Health Quality 
Ontario 


Let’s make our health system healthier 


ONTARIO HEALTH TECHNOLOGY 
ASSESSMENT SERIES 


Internet-Delivered Cognitive Behavioural Therapy for Major 
Depression and Anxiety Disorders: A Health Technology 
Assessment 


KEY MESSAGES 


What Is This Health Technology Assessment About? 


Major depression is characterized by depressed mood and diminished interest or pleasure in all, or 
almost all, activities. Anxiety disorders are characterized by feelings of distress and fear that interfere with 
day-to-day functioning. Cognitive behavioural therapy (CBT) is a form of evidence-based psychotherapy 
used to treat major depression and anxiety disorders. Internet-delivered CBT (iCBT) is structured CBT 
delivered via the internet. Guided iCBT involves communication with a regulated health care professional, 
whereas unguided iCBT is provided without the support of a regulated health care professional. 


This health technology assessment looked at how safe, effective, and cost-effective guided and unguided 
iCBT are for the treatment of adults with mild to moderate major depression or anxiety disorders. We also 
looked at the budget impact of publicly funding iCBT and the preferences, values, and experiences of 
people with mild to moderate major depression or anxiety disorders. 


What Did This Health Technology Assessment Find? 


Compared with waiting list, guided iCBT improves symptoms of mild to moderate major depression and 
select anxiety disorders. 


Guided iCBT represents the most economical option for the short-term treatment of adults with mild to 
moderate major depression or anxiety disorders. Over the next 5 years, we estimate that publicly funding 
guided iCBT will result in additional annual costs of between $10 million and $40 million for mild to moderate 
major depression and between $16 million and $65 million for anxiety disorders. 


People with anxiety or depression with whom we spoke viewed iCBT as an effective treatment option. 
Internet-delivered CBT may be especially beneficial for those whose symptoms may prevent them from 
leaving home. However, participants reported important barriers and limitations to using iCBT, including 
the need for a computer, internet access, and computer literacy, as well as the ability to understand 
complex written information. Participants found that the cost of treatment, the number of sessions in a 
course of treatment, and the lack of follow-up support were also substantial drawbacks for iCBT. 
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ABSTRACT 


Background 


Major depression is defined as a period of depression lasting at least 2 weeks characterized by 
depressed mood, most of the day, nearly every day, and/or markedly diminished interest or 
pleasure in all, or almost all, activities. Anxiety disorders encompass a broad range of disorders 
in which people experience feelings of fear and excessive worry that interfere with normal day- 
to-day functioning. 


Cognitive behavioural therapy (CBT) is a form of evidence-based psychotherapy used to treat 
major depression and anxiety disorders. Internet-delivered CBT (iCBT) is structured, goal- 
oriented CBT delivered via the internet. It may be guided, in which the patient communicates 
with a regulated health care professional, or unguided, in which the patient is not supported by a 
regulated health care professional. 


Methods 


We conducted a health technology assessment, which included an evaluation of clinical benefit, 
value for money, and patient preferences and values related to the use of iCBT for the treatment 
of mild to moderate major depression or anxiety disorders. We performed a systematic review of 
the clinical and economic literature and conducted a grey literature search. We reported 

Grading of Recommendations Assessment, Development, and Evaluation (GRADE) ratings if 
sufficient information was provided. When other quality assessment tools were used by the 
systematic review authors in the included studies, these were reported. We assessed the risk of 
bias within the included reviews. We also developed decision-analytic models to compare the 
costs and benefits of unguided iCBT, guided iCBT, face-to-face CBT, and usual care over 

1 year using a sequential approach. We further explored the lifetime and short-term cost- 
effectiveness of stepped-care models, including iCBT, compared with usual care. We calculated 
incremental cost-effectiveness ratios (ICERs) from the perspective of the Ontario Ministry of 
Health and Long-Term Care and estimated the 5-year budget impact of publicly funding iCBT 
for mild to moderate major depression or anxiety disorders in Ontario. To contextualize the 
potential value of iCBT as a treatment option for major depression or anxiety disorders, we 
spoke with people with these conditions. 


Results 


People who had undergone guided iCBT for mild to moderate major depression (standardized 
mean difference [SMD] = 0.83, 95% Cl 0.59-1.07, GRADE moderate), generalized anxiety 
disorder (SMD = 0.84, 95% Cl 0.45—1.23, GRADE low), panic disorder (small to very large 
effects, GRADE low), and social phobia (SMD = 0.85, 95% Cl 0.66—1.05, GRADE moderate) 
showed a statistically significant improvement in symptoms compared with people on a waiting 
list. People who had undergone iCBT for panic disorder ( SMD= 1.15, 95% Cl: 0.94 to 1.37) and 
iCBT for social anxiety disorder (SMD=0.91, 95% Cl: 0.74—1.07) showed a statistically 
significant improvement in symptoms compared with people on a waiting list. There was a 
statistically significant improvement in quality of life for people with generalized anxiety disorder 
who had undergone iCBT (SMD = 0.38, 95% Cl 0.08—0.67) compared with people on a waiting 
list. The mean differences between people who had undergone iCBT compared with usual care 
at 3, 5, and 8 months were -4.3, -3.9, and -5.9, respectively. The negative mean difference at 
each follow-up showed an improvement in symptoms of depression for participants randomized 
to the iCBT group compared with usual care. People who had undergone guided iCBT showed 
no statistically significant improvement in symptoms of panic disorder compared with individual 
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or group face-to-face CBT (d = 0.00, 95% Cl -0.41 to 0.41, GRADE very low). Similarly, there 
was no statistically significant difference in symptoms of specific phobia in people who had 
undergone guided iCBT compared with brief therapist-led exposure (GRADE very low). There 
was a small statistically significant improvement in symptoms in favour of guided iCBT 
compared with group face-to-face CBT (d= 0.41, 95% Cl 0.03—0.78, GRADE low) for social 
phobia. There was no statistically significant improvement in quality of life reported for people 
with panic disorder who had undergone iCBT compared with face-to-face CBT (SMD = -0.07, 
95% Cl -0.34 to 0.21). 


Guided iCBT was the optimal strategy in the reference case cost-utility analyses. For adults 
with mild to moderate major depression, guided iCBT was associated with increases in both 
quality-adjusted survival (0.04 quality-adjusted life-years [QALYs]) and cost ($1,257), yielding 
an ICER of $31,575 per QALY gained when compared with usual care. In adults with anxiety 
disorders, guided iCBT was also associated with increases in both quality-adjusted survival 
(0.03 QALYs) and cost ($1,395), yielding an ICER of $43,214 per QALY gained when compared 
with unguided iCBT. In this population, guided iCBT was associated with an ICER of $26,719 
per QALY gained when compared with usual care. The probability of cost-effectiveness of 
guided iCBT for major depression and anxiety disorders, respectively, was 67% and 70% at 
willingness-to-pay of $100,000 per QALY gained. Guided iCBT delivered within stepped-care 
models appears to represent good value for money for the treatment of mild to moderate major 
depression and anxiety disorders. 


Assuming a 3% increase in access per year (from about 8,000 people in year 1 to about 32,000 
people in year 5), the net budget impact of publicly funding guided iCBT for the treatment of mild 
to moderate major depression would range from about $10 million in year 1 to about $40 million 
in year 5. The corresponding net budget impact for the treatment of anxiety disorders would 
range from about $16 million in year 1 (about 13,000 people) to about $65 million in year 5 
(about 52,000 people). 


People with depression or an anxiety disorder with whom we spoke reported that iCBT improves 
access for those who face challenges with face-to-face therapy because of costs, time, or the 
severity of their condition. They reported that iCBT provides better control over the pace, time, 
and location of therapy, as well as greater access to educational material. Some reported 
barriers to iCBT include the cost of therapy; the need for a computer and internet access, 
computer literacy, and the ability to understand complex written information. Language and 
disability barriers also exist. Reported limitations to iCBT include the ridigity of the program, the 
lack of face-to-face interactions with a therapist, technological difficulties, and the inability of an 
internet protocol to treat severe depression and some types of anxiety disorder. 


Conclusions 


Compared with waiting list, guided iCBT is effective and likely results in symptom improvement 
in mild to moderate major depression and social phobia. Guided iCBT may improve the 
symptoms of generalized anxiety disorder and panic disorder compared with waiting list. 
However, we are uncertain about the effectiveness of iCBT compared with individual or group 
face-to-face CBT. Guided iCBT represents good value for money and could be offered for the 
short-term treatment of adults with mild to moderate major depression or anxiety disorders. 
Most people with mild to moderate depression or anxiety disorders with whom we spoke felt 
that, despite some perceived limitations, iCBT provides greater control over the time, pace, and 
location of therapy. It also improves access for people who could not otherwise access therapy 
because of cost, time, or the nature of their health condition. 
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OBJECTIVE 


This health technology assessment looked at the effectiveness, safety, cost-effectiveness, 
budget impact of publicly funding, and patient preferences and values associated with internet- 
delivered cognitive behavioural therapy (iCBT) for the treatment of mild to moderate major 
depression and anxiety disorders. 


This health technology assessment has been registered on PROSPERO, the international 
prospective register of systematic reviews (CRD42018096042), available at 


https://www.crd.york.ac.uk/PROSPERO. 


BACKGROUND 


Health Condition 


Major depression is one of the most common mental illnesses, imposing a huge human and 
economic burden on people and society. Each year, about 7% of people in Canada meet the 
diagnostic criteria for major depression, and about 13% to 15% of these people will experience 
major depression for the rest of their lives.' The essential feature of major depression is the 
occurrence of one or more major depressive episodes, defined as periods lasting at least 

2 weeks characterized by depressed mood most of the day, nearly every day, and/or a 
markedly diminished interest or pleasure in all, or almost all, activities. To receive a diagnosis 
of major depression, within the same 2-week period a person must experience five or more 
symptoms from the criteria for a major depressive episode as described in the current version of 
the Diagnostic and Statistical Manual of Mental Disorders (DSM).* 


Major depression is both chronic (lasting 3 months or more) and episodic (consisting of 
separate episodes) in nature. It consists of initial phases (i.e., the acute and continuation 
phases, each lasting approximately 3 months) and a maintenance phase (lasting approximately 
6 to 24 months, with an average 9 to 12 months).*:*® The category of anxiety disorders includes 
a broad range of disorders, including generalized anxiety disorder (GAD), panic disorder (with 
and without agoraphobia), social anxiety disorder, and specific phobias (common phobias 
include fear of animals, insects, germs, heights, thunder, driving, public transportation, flying, 
dental or medical procedures, and elevators). People with anxiety disorders experience feelings 
of fear and excessive worry that impacts their overall well-being and functioning. The DSM 
classifies post-traumatic stress disorder (PTSD) and obsessive-compulsive disorder (OCD) 
outside of the category of anxiety disorders.”® Anxiety disorders can exist in isolation or coexist 
with other anxiety and depressive disorders.%'° 


Clinical Need and Target Population 


Approximately 11.3% of the Canadian population have been classified as meeting criteria for 
major depression at some point in their life.2"' Major depression affects not only individuals and 
families but also occupational functioning, through absenteeism and presenteeism (loss of 
productivity from attending work while unwell).2 Major depression also negatively affects 
people’s ability to perform personal activities such as parenting and housekeeping. 


As of 2006, the lifetime prevalence rates of panic disorder, agoraphobia, and social phobia in 
Canada were 3.7%, 1.5%, and 8.1%, respectively.'* One Ontario study estimated that 12% of 
adults between the ages of 15 and 64 years—9% of men and 16% of women—experience an 
anxiety disorder during any 12-month period. '2 
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Current Treatment Options 


Treatment for acute major depression consists of pharmacological and psychological 
interventions. The use of antidepressant medications has increased over the last 20 years, 
mainly due to the advent of selective serotonin reuptake inhibitors, as well as newer agents. '? 
While antidepressants continue to be a mainstay in the treatment of major depression, 
adherence rates are low, in part because of patients’ concerns about side effects and possible 
dependency. Surveys have demonstrated patients’ preference for psychological therapies over 
antidepressants. '° 


Psychotherapy is the treatment of mental or emotional illness through psychological methods 
rather than through drugs. There are many types of psychotherapy. Cognitive behavioural 
therapy (CBT) is an evidence-based, structured, intensive, time-limited, symptom-focused form 
of psychotherapy recommended for the treatment of major depression and anxiety disorders. '* 


Cognitive behavioural therapy helps people become aware of how certain negative automatic 
thoughts, attitudes, expectations, and beliefs contribute to feelings of sadness and anxiety. 
People undergoing CBT learn how their thinking patterns, which may have developed in the 
past to deal with difficult or painful experiences, can be identified and changed to reduce 
unhappiness. 4 


Barriers to face-to-face CBT include stigmas around people seeking help in person, geography 
(distance from health care professional), time, and cost. Increasingly, there is a desire to pursue 
internet delivery as an option to increase access to treatment. '® 


The treatment of major depression can be divided into acute and maintenance phases.2*° The 
aim of treatment in the acute and continuation phases is the reduction or elimination (remission) 
of symptoms and a return to the level of psychological and social functioning (psychosocial 
functioning) experienced before the onset of major depression.” The aim of treatment in the 
maintenance phase is to prevent symptoms from recurring.” 


A Health Quality Ontario quality standard on major depression' recommends that people with 
major depression have timely access to either antidepressant medication or evidence-based 
psychotherapy, based on their preference. Clinical guidelines suggest that CBT may be offered 
as an initial treatment for anxiety disorders. Pharmacological treatment may be considered if the 
person has a poor response to CBT treatment.'® 


The National Institute for Health and Care Excellence (NICE) guidelines recommend a stepped- 
care approach that starts with low-intensity treatments such as guided iCBT for people with mild 
to moderate major depression or anxiety disorders.'’ They recommend higher-intensity 
interventions (face-to-face psychotherapies such as CBT or interpersonal therapy) alone or in 
combination with medications for people who do not respond to treatment or who progress to 
more severe depression or anxiety.'” 


Health Service Under Review 


Internet-delivered CBT is based on the principles of CBT, consists of structured modules with 
clearly defined goals, and is delivered via the internet.'* Although there are many types of iCBT 
programs, each are goal oriented sessions that typically consist of 8 to 12 modules and can be 
guided or unguided." Internet-delivered CBT programs are made available by computer, 
smartphone, or tablet, for a fee.'* With unguided iCBT, patients are informed of a website 
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through which they can participate in an online self-directed program. Guided iCBT involves 
support from a regulated health professional (e.g., social worker, psychologist, psychotherapist, 
occupational therapist, nurse, or physician). In guided iCBT, people complete modules and 
communicate (via email, text messages, or telephone calls) their progress to a regulated health 
professional. '* 


Current recommendations indicate that iCBT is not appropriate for severely ill people." 


Regulatory Information 
Internet-delivered CBT does not require regulatory approval from Health Canada. 
Ontario Context 


Internet-delivered CBT is not currently publicly funded in a systematic manner in Ontario. 
Guided iCBT is currently provided by some hospitals and in the private sector. There are 
several pilot programs underway or recently completed in Canada, funded through public and 
private sources. 
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CLINICAL EVIDENCE 


Research Question 


What are the effectiveness and safety of iCBT for improving outcomes for adults with mild to 
moderate major depression or anxiety disorders? 


Methods 


We developed the research questions in consultation with health care providers and clinical 
experts. 


Clinical Literature Search 


We performed a literature search on February 15, 2018, to retrieve studies published from 
January 1, 2000 to the search date. We used the Ovid interface to search the following 
databases: MEDLINE, Embase, Cochrane Database of Systematic Reviews, Health Technology 
Assessment, National Health Service Economic Evaluation Database (NHS EED), and 
PsycINFO. We used the EBSCOhost interface to search the Cumulative Index to Nursing & 
Allied Health Literature (CINAHL). 


Medical librarians developed the search strategies using controlled vocabulary (e.g., Medical 
Subject Headings) and relevant keywords. We applied a search filter to limit results to 
systematic reviews, meta-analyses, and health technology assessments. The final search 
strategy was peer reviewed using the PRESS Checklist.'® We created database auto-alerts in 
MEDLINE, Embase, PsycINFO, and CINAHL and monitored them for the duration of the 
assessment period. 


We performed targeted grey literature searching of health technology assessment agency 
websites and PROSPERO. See Appendix 1 for the literature search strategies, including all 
search terms. 


Literature Screening 


Two reviewers conducted an initial screening of titles and abstracts using Covidence 
management software and obtained full-text articles that appeared eligible according to the 
inclusion criteria. The reviewers then examined the full texts of articles that appeared eligible to 
identify studies eligible for inclusion. 


Inclusion and Exclusion Criteria 
Studies 


We conducted an overview of systematic reviews of randomized controlled trials. We 
considered publications to be systematic reviews if they met all the following criteria: 


e Clearly described inclusion and exclusion criteria 
e Undertook a reproducible search of two or more electronic literature databases 
e Assessed and documented the quality of the included randomized controlled trials 


We included English-language full-text systematic reviews of randomized controlled trials 
published between January 1, 2000, and February 15, 2018. 
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Participants 


We included studies of outpatient adults aged 16 years and older with a primary diagnosis of 
mild to moderate major depression or anxiety disorder according to validated diagnostic 
instruments such as the DSM, the International Statistical Classification of Diseases and 
Related Health Problems (ICD), the Centre for Epidemiological Scale for Depression, the Beck 
Depression Inventory, or the Patient Health Questionnaire, Structured Diagnostic Interview 
Schedule.' We included studies of people with a primary diagnosis of anxiety disorder or of mild 
to moderate major depression coexisting with other mental health conditions (excluding OCD 
and PTSD). 


We excluded studies of people less than 16 years old or had participants with postpartum 
depression, bipolar disorder, dysthymia, seasonal affective disorder, a psychotic disorder, drug 
or alcohol dependence-related depression or anxiety, obsessive compulsive disorder, 
posttraumatic stress disorder, major depression or anxiety comorbid with physical disorders 
(e.g., cancer, stroke, or acute coronary syndrome). 


Intervention 


We included reviews that assessed iCBT. We excluded non-traditional CBT (e.g., mindfulness 
CBT), transdiagnostic interventions, CBT delivered via bibliotherapy, and CBT described as 
computerized but for which there was no analysis specifically for iCBT. 


Comparators 


We included the following comparators: 


Face-to-face CBT, defined as individual or group face-to-face CBT 

Usual care, defined as any treatment prescribed by a general practitioner 
Waiting list, defined as participants receiving iCBT at a later date 
Combination of usual care, waiting list, and/or information control 


Outcomes of Interest 


We included the following outcomes of interest: 


Remission of depression or anxiety symptoms (acute phase) 

Prevention of relapse following a successful acute treatment (maintenance phase) 
Response to therapy (50% reduction in symptoms from baseline) 

Safety 

Quality of life 

Satisfaction with care 

Patient adherence 


Data Extraction 


Two reviewers extracted relevant data using a data extraction form that included the following 
study characteristics: study population, inclusion and exclusion criteria, description of the 
interventions, types of comparators, outcomes, results, and quality assessment as conducted by 
authors of the systematic reviews. For reviews where a portion of the participants or the 
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intervention did not match the population or intervention of interest, we extracted the results 
specific to our population or intervention of interest 


We contacted authors of the systematic reviews to provide clarification as needed. 


Evidence Synthesis 


We undertook a narrative summary of the results reported in the included systematic reviews. 
We did not perform an analysis of primary studies. Results for guided and unguided iCBT were 
reported separately where available. Unless specified, “iCBT” refers to both guided and 
unguided iCBT. 


Critical Appraisal of Evidence 


A single reviewer assessed risk of bias using the Risk of Bias in Systematic Reviews (ROBIS) 
tool.'? See Appendix 2 for details of the ROBIS assessment. 


We assessed the quality of the evidence within the included reviews by extracting the review 
authors’ Grading of Recommendations Assessment, Development, and Evaluation (GRADE) 
ratings if sufficient information was provided. If other quality assessment tools were used by the 
systematic review authors in the included studies, these were reported. 


Expert Consultation 


Consulted experts included physicians in the specialty areas of psychiatry and psychology and 
regulated mental health professionals. Their role was to review the clinical review plan, 
contextualize the evidence, and provide feedback on the appropriate use of iCBT. 
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Results 


Literature Search 


The literature search yielded 393 citations published between January 1, 2000, and February 


February 2019 


15, 2018, after removing duplicates. We obtained the full text of 148 articles for further 


assessment. Seven systematic reviews met the inclusion criteria.”*°*> The primary reasons for 
exclusions are provided below. See Appendix 3 for a selected list of studies excluded after full- 


text review that includes the primary reason for exclusion. 


Figure 1 presents the flow diagram for the Preferred Reporting Items for Systematic Reviews 


and Meta-analyses (PRISMA). 


Records identified through 
database searching (n = 770) 


Identification 


Records screened 
(n = 393) 


Full-text articles assessed 
for eligibility (n = 143) 


> 
LS 
re 
fey 
i 


Studies included in 
qualitative synthesis 
(n= 7) 


Studies included in 
quantitative synthesis 
(meta-analysis) (n = 0) 


Additional records identified through 


other sources (n = 13) 


Records after duplicates removed 


Records excluded 
(n = 250) 


Full-text articles excluded (n = 136) 


No quality assessment (26) 

No subgroup analyses (17) 

Wrong study design (20) 

No details provided on how major depression 
and/ or anxiety was diagnosed (15) 

No details provided on mild to moderate major 
depression (10) 

Duplicate study (11) 

Wrong intervention (11) 

Unable to extract data for 16 years and older 
(9) 

Did not use at least 2 electronic databases (7) 
Wrong research question (4) 

Wrong comparator (2) 

Wrong outcome (1) 

Full text unavailable (2) 

No details on primary anxiety diagnosis (1) 


Figure 1: PRISMA Flow Diagram—Clinical Search Strategy 


Source: Adapted from Moher et al.?6 
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Characteristics of Included Systematic Reviews 


We identified one systematic review”? that evaluated iCBT for mild to moderate major 
depression and five systematic reviews’?':23-5 that evaluated iCBT for anxiety disorders. One 
systematic review reported data on both mild to moderate major depression and anxiety 
disorders.”° There was inconsistency across the systematic reviews in reporting and analyzing 
the level of support associated with iCBT. One systematic review reported the degree of support 
as therapist-guided (i.e., clinical support).2° The type of therapist support included email 
correspondence and weekly phone conversations of 10 to 20 minutes for each participant. 


The instruments used to diagnose mild to moderate major depression and anxiety disorders in 
the inclusion criteria of the included systematic reviews varied. There was additional variation 
across the scales used to measure the change in symptoms from pre- to post-treatment. 


Characteristics of the included systematic reviews are summarized in Table 1. 
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Table 1: Characteristics of Included Systematic Reviews 


February 2019 


Author, Year Objective Study Design and Methods Comparators Outcomes of interest 
Andrews et al, Replication and extension of Inclusion criteria Waitlist control, information Subgroup analyses examining the 
20187! 2010 meta-analysis to Adutls 218 yrs with a primary diagnosis of either major control, care as usual, or effects of iCBT on change in — 
examine whether depression, GAD, PD with or without agoraphobia, or SAD placebo. symptom severity for depression 
computerised therapy for the : . . ee and anxiety disorders. 
anxiety and depressive Diagnosis could be determined by a clinician, through 


disorders is effective telephone interview or by meeting a recognised cut-off on a 

acceptable, and practical validated self-report questionnaire. 

health care. Exclusion criteria 
Studies of treatments aimed at a range of diagnoses 
(transdiagnostic studies), and studies of depressive or 
anxiety symptoms in which no data on the probability of 
satisfying diagnostic criteria were supplied. 


Arnberg et al, 2014° (1) Is internet-delivered Inclusion Criteria 


Any established psychological 


psychological treatment Participants: children, adolescents, and adults with anxiety or treatments, waiting list, usual 


efficacious, safe, and cost- mood disorders, including major depression, dysthymia, 
effective for mood and anxiety bipolar disorder, social phobia, PD, GAD, PTSD, OCD, 


disorders in children, . specific phobia, and separation anxiety (in children and 
adolescents and adults? adolescents). 


(2) Is internet-delivered Intervention: internet-delivered psychological treatments that 
treatment noninferior to are based on an explicit psychological theory and not 
established psychological conducted at a clinic. Any support had to be remotely 
treatments? delivered (e.g. email-like messages or telephone). The 
degree of support was categorized into pure self-help (no 
support), technician-assisted (e.g., nonclinical), 
or therapist-guided (i.e., clinical support). 
Study design: for short-term effects and risk of adverse 
events, only RCTs were included. For long-term follow-up 
assessments (i.e., 26 mo post-assessment), RCTs and 
observational studies were included because of the ethical 
and practical dilemmas of conducting long-term RCTs. For 
cost-effectiveness data, economic evaluations based on 
individual-level data and decision models were used. 


Exclusion Criteria 


Studies where the participants were selected primarily 
because of a specific physical illness. 


care, or attention control. 


Change in symptoms of the 
primary disorder, adverse events, 
and cost per effect and per 
quality-adjusted life year. 
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Author, Year 


Objective 


Study Design and Methods 


February 2019 


Comparators Outcomes of interest 


Kampmann et al, 
20167 


Adelman et al, 2014’ 


Kaltenthaler et al, 
200872 


Evaluate the efficacy of 
technology-assisted 
interventions for individuals 
with a diagnosis of SAD. 


Examine cCBT efficacy for 
non-PTSD, non-OCD anxiety 
disorders along multiple 
dimensions, including 
treatment efficacy by 
comparison condition, 
diagnostic target, level of 
therapist involvement, study 
quality, and participant age 
group. 


Systematically review RCTs 
of computerized CBT (cCBT) 
software packages for the 
treatment of mild to moderate 
depression. 


Inclusion Criteria 


Participants: Its 218 yrs who meet the criteria for a diagnosis 
of SAD and who had their SAD symptoms assessed during 
or after the initial assessment. 

Interventions: treatments targeting SAD symptoms. 


Study design: RCTs with at least 10 participants per 
treatment condition, no language restrictions. 


Exclusion Criteria 
Dissertation abstracts, reviews, and study protocols. 


Inclusion Criteria 


RCTs assessing efficacy of cCBT for anxiety disorders, 
subjects who meet criteria for GAD, PD, SAD, or a specific 
phobia based on DSM-IV criteria, trials recorded to compare 
cCBT to waiting list or in-person CBT control condition. 


Exclusion Criteria 

OCD and PTSD trials, due to the underlying neuropathology 
of these conditions, the CBT techniques used to treat these 
conditions were considered to be sufficiently different from 
the other anxiety disorders, OCD, and PTSD within 
diagnostic categories that are distinct from the anxiety 
disorders . Trials with <10 participants. 


Inclusion Criteria 


Adults with mild to moderate depression, with or without 
anxiety, as defined by individual studies. 


Exclusion Criteria 

Studies on postnatal depression, bipolar disorder, depression 
with psychotic symptoms or current major depression, or 
serious suicidal thoughts. 


Passive control, active control. Symptoms of depression at post- 
assessment, efficacy and 


changes in quality of life. 


Endpoint score on a rating scale 
used to measure anxiety. 


Results were stratified by 
comparator. 


Waiting list or in-person CBT. 


Current standard treatments 
including therapist-led CBT, 
non-directive counselling, 
primary care counselling, 
routine management (including 
drug treatment), and 
alternative methods of CBT 
delivery such as bibliotherapy 
and group CBT. 


Improvement in psychological 
symptoms, quality of life, patient 
satisfaction. 
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Author, Year 


Objective Study Design and Methods Comparators 


February 2019 


Outcomes of interest 


Dedert et al, 201322 (1) For adults with depressive Inclusion Criteria Usual care not involving 


disorder, PTSD, PD, or GAD, participants: Adults 218 yrs with one or more of the following Psychotherapy; waitlist control; 


what are the effects of CCBT — ongitions: attention/information control, 

interventions compared with : : é : ; cCBT with a different level of 

inactive controls? * Unipolar depressive disorder (major depression, therapist support, face-to-face 
i ; dysthymia, minor depression, adjustment disorder with 

(2) For cCBT interventions, depressed mood, or mixed anxiety/depression). j 


what level, type, and modality 


of user support is provided ¢ Posttraumatic stress disorder. 


(e.g., daily telephone calls, * GAD, PD, and anxiety disorder not otherwise specified. 
weekly email * Score above the threshold for significant depressive or 
correspondence), who anxiety symptoms using a validated questionnaire. 


rovides this support (e.g., ; eet : 
heageh ee See * Comorbid psychiatric disorder, as long as the primary 


peer), what is the clinical disorder is a condition of interest. 
context (primary intervention, | * In mixed samples of children and adults, at least 80% 


adjunct), and how is this must be 218 yr (or the mean age minus 1.5 SD 218 yrs). 
support pelaled to patient + In studies that include patients with a large number of 
outcomes? Examine the conditions, at least 80% must have one of the conditions 
influence of support-related of interest. 


factors on treatment 


outcomes, including Interventions: CBT delivered primarily by a computerized 


satisfaction, response, and (i.e., electronic) mechanism. Interventions may be self-guided 
; : or with clinician support, but the computerized mechanism 


letion. ‘ : ; 
pelt wba ; : must be the key intervention that differs from the control 
(3) For adults with major group. 


depression, PTSD, PD, or : ‘ 

GAD, MIRET ERD Tie QEGS Gi Study design: RCTs with N > 20. 

cCBT interventions compared 

with oar ead : Exclusion Criteria 

moe alae aa Participants: people with test anxiety, phobias, or SAD. 
Interventions: interpersonal therapy designed to prevent the 
onset or relapse of mental illness; interventions that are 
primarily telemedicine-based (e.g., therapy via video chat or 
phone interactions, including those by interactive voice 
response); interventions that use virtual reality as the primary 
therapeutic mode, do not use the key components of CBT, 
disease management interventions where CBT is only one 
component of a more comprehensive intervention, are 
delivered primarily in face-to-face encounters but 
supplemented by text messages, or use online materials that 
do not meet the definition of CBT or CBT-related intervention. 


Patient satisfaction, safety, 
symptom measure, health related 
quality of life. 
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Author, Year Objective Study Design and Methods Comparators Outcomes of interest 
Richards et al, Systematically review and Inclusion Criteria Waiting list Clinical efficacy 
24 i 
2015 conduct a meta-analysis of = aquits >18yr who have a clinical diagnosis of GAD and may 
internet-delivered have comorbidity with depression and/or impairment in 
psychological therapy for functioning. 


GAD compared to waiting list 


control groups. Study design: RCTs. 


Abbreviations: cCBT, computerized CBT; GAD, generalized anxiety disorder; iCBT, internet-delivered cognitive behavioural therapy; OCD, obsessive compulsive disorder; PD, panic disorder; PTSD, 
posttraumatic stress disorder; RCT, randomized controlled trial; SAD, social anxiety disorder; SD, standard deviation. 
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Mild to Moderate Major Depression 
Guided Internet-Delivered CBT Compared to Waiting List 
Symptoms and Response to Treatment 


Arnberg et al?° identified five randomized control trials comparing guided iCBT to waiting list. 
People on the waiting list received iCBT after participants randomized to receive iCBT 
completed the study. Guided iCBT was delivered in the form of email correspondence or 
telephone conversations lasting about 10 to 20 minutes for each participant on a weekly basis. 
Participants who received guided iCBT experienced a reduction in symptoms of depression 
compared with waiting list (standardized mean difference [SMD] = 0.83, 95% confidence interval 
[Cl] 0.59-1.07). This reduction was statistically significant. 


Internet-Delivered CBT Compared to Weekly Telephone Calls 
Symptoms and Response to Treatment 


Kaltenthaler et al? reported data from one randomized controlled trial that assessed changes in 
psychological outcomes before and 6 weeks after treatment for participants with mild to 
moderate major depression randomized to an iCBT program or control group. The study did not 
specify whether participants received guided or unguided iCBT. The control group received 
phone calls from interviewers once a week to discuss lifestyle and environmental factors that 
may have had an influence on depression. The authors found that participants who received 
iCBT showed a significant reduction in symptoms of depression compared to the control group 
(mean difference 3.2, 95% Cl 0.9—5.4). This randomized controlled trial used the Centre for 
Epidemiologic Studies Depression Scale as a primary outcome measure of symptoms of 
depression. 


Patient Satisfaction 


Kaltenthaler et al?? reported that participants overall accepted the iCBT program. 


Patient Dropout and Treatment Adherence 


Kaltenthaler et al? analyzed one randomized controlled trial that reported on patient dropout 
when iCBT was compared with participants who received weekly telephone calls from 
interviewers about environmental and lifestyle factors that may have had an influence on 
depression. Of 525 participants randomized, 25.3% dropped out of the iCBT program, and 10% 
of participants who received weekly telephone calls from interviewers about environmental and 
lifestyle factors were loss to follow-up. 


Internet-Delivered CBT Compared to Usual Care 
Symptoms and Response to Treatment 


Kaltenthaler et al? reported data from a randomized controlled trial that assessed change in 
psychological outcomes before and after treatment for participants with mild to moderate major 
depression randomized to an iCBT program or usual care. Usual care was defined as whatever 
treatment the general practitioner prescribed, such as medications, or referral to a counsellor or 
a health professional. The authors found that the mean score using the Beck Depression 
Inventory at 3, 5, and 8 months after treatment demonstrated an improvement in symptoms for 
participants receiving iCBT compared to usual care. The mean difference between iCBT and 
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usual care was —4.3, -3.9, and -5.9 at 3, 5, and 8 months, respectively.” The negative mean 
difference at each follow-up signifies improvement in symptoms of depression for participants 
randomized to the iCBT group compared to usual care. 


Patient Satisfaction 


Kaltenthaler et al?? reported that participants receiving iCBT were significantly more satisfied 
with treatment compared to individuals randomized to usual care. 


Patient Dropout and Treatment Adherence 


Kaltenthaler et al?? reported data from a randomized controlled trial that assessed patient drop 
out for participants with mild to moderate major depression randomized to an iCBT program or 
usual care. Of the 274 participants, 35% dropped out during the study.”* The proportion of 
participants who were lost to follow-up in the iCBT program versus the usual care program is 
unclear. 


See Table 2 for summary of results for major depression. 
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Table 2: Summary of Results of Included Systematic Reviews for Mild to Moderate Major 
Depression 


No. of Studies/ 
Author, Year No. of Participants Results Quality Assessment 


Guided internet-deliverd CBT compared with waiting list 


Symptoms and Response to Treatment 


Arnberg et al, 5 RCTs/159 SMD = 0.83 (95% Cl 0.59-1.07) GRADE: ®®® moderate 
20 
2014 


Internet-delivered CBT compared with weekly telephone calls 


Symptoms and Response to Treatment 


Kaltenthaler et 1 RCT/525 SMD = 3.2 (95% Cl 0.95.4) CASP Tool 
al, 200822 Randomization method: statistical software 
program used 
Masking: no maskedAssessment 
Power calculation: yes 
Loss to follow-up loss: number and some 


reasonsreported 
Patient Dropout and Treatment Adherence 
Kaltenthaler et 1 RCT/525 25.3% of participants dropped out of | CASP Tool 
al, 200822 the iCBT program;10% of Randomization method: statistical software 
participants dropped out from the program used 
control group, who received weekly Masking: no masked assessment 
telephone calls Power calculation: yes 


Loss to follow-up loss: number and some 
reasons reported 


Internet-delivered CBT compared with usual care 


Symptoms and Response to Treatment 


Kaltenthaler et 1 RCT/274 iCBT vs. usual care CASP Tool 
al, 200822 3 mos: -4.3 (95% Cl not reported)? Randomization: sealed envelopes, stratified 
5 mos: -3.9 (95% Cl not reported)? for medication and duration of current episode 
8 mos: -5.9 (95% Cl not reported)? Masking: no masked assessment 
Power calculation: yes 
Loss to follow-up: number and some reasons 


reported 
Patient Dropout and Treatment Adherence 
Kaltenthaler et 1 RCT/274 35% dropped out during the study CASP Tool 
al, 200822 Randomization: SPSS function 


Masking: no masked assessment 
Power calculation: yes 
Loss to follow-up: loss and reasons reported 


Abbreviations: Cl, confidence interval; iCBT, internet-delivered cognitive behavioural therapy; SMD, standardized mean difference. 
A positive improvement in symptoms in favour of iCBT. The mean differences were calculated by the authors of the health technology assessment. 


Anxiety Disorders 
Internet-Delivered CBT Compared to Waiting List 
Symptoms and Response to Treatment 


Arnberg et al?° identified four randomized controlled trials that reported on changes in symptoms 
for participants with generalized anxiety disorder. A large statistically significant pooled effect 
was found for guided iCBT that involved therapist (i.e., clinical) support compared to waiting list 
(SMD = 0.84, 95% Cl 0.45-1.23). 
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Similarly, Adelman et al’ identified four randomized controlled trials that reported on anxiety 
symptoms for generalized anxiety disorder. There was a large statistically significant 
standardized mean difference that showed an improvement in anxiety symptoms for participants 
who received iCBT compared to waiting list (SMD = 1.06, 95% Cl 0.82—1.30). 


Arnberg et al?° found small to very large effects in favour of guided iCBT, which involved 
therapist (i.e., clinical) support, compared to waiting list for individuals with panic disorder.** 


Adelman et al’ identified eight randomized controlled trials that reported on panic disorder. The 
authors found a large statistically significant standardized mean difference showing an 
improvement in anxiety symptoms for participants who received iCBT compared to waiting list 
(SMD = 1.15, 95% Cl 0.94— 1.37). 


Arnberg et al?° reported on changes in symptoms for eight randomized controlled trials on social 
phobia that compared guided iCBT, which involved therapist (i.e., clinical) support, to waiting 
list. A large statistically significant pooled effect was found in favour of guided iCBT compared to 
waiting list (SMD = 0.85, 95% Cl 0.66-1.05). 


Adelman et al’ identified nine randomized controlled trials that reported on anxiety symptoms for 
social anxiety disorder. The authors found a large statistically significant standardized mean 
difference that showed improvement in anxiety symptoms for participants who received iCBT 
compared to waiting list (SMD = 0.91, 95% Cl 0.74-1.07). 


Dedert et al?? identified four trials (five comparisons) in people with generalized anxiety disorder 
and found that those who had undergone iCBT experienced a large statistically significant 
difference in symptoms of anxiety compared with people on a waiting list (SMD -0.94, 

95% Cl -1.34 to -0.54). 


Quality of Life 


Richards et al?4 identified two studies on generalized anxiety disorder that reported on the 
difference in quality of life using the Quality-of-Life Inventory pre- and post-assessment between 
people receiving iCBT compared to waiting list. Particiapnts that received iCBT had a 
statistically significant improvement in quality of life at post assessment compared to waiting list 
(SMD = 0.38, 95% Cl 0.08-0.67). 


Patient Dropout and Treatment Adherence 


Dedert et al?? reported treatment adherence as the percentage of patients completing all 
planned sessions stratified by the level of support. The authors identified two randomized 
controlled trials that reported on treatment adherence for generalized anxiety disorder, and 
there was variation based on the level of support provided alongside iCBT. Support provided 
alongside iCBT included feedback by a technician (nonlicensed staff) or clinician (licensed 
professional), based on the participant’s previous interactions with the program, and 
psychoeducation. This type of support was delayed (not live). Live support comprised phone 
sessions, a scheduled chat on internet forums, or instant messaging with either technicians or 
Clinicians. Seventy-five percent of participants completed all sessions when there was live 
support, compared to only 11% when support was provided alongside iCBT.*° These results 
suggest that participants who received instant communication with either technicians or 
clinicians completed more sessions than those who received delayed communication. 
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Internet-Delivered CBT Compared to a Combination of Usual Care, Waiting List, 
and/or Information Control 


Symptoms and Response to Treatment 


Andrews et al?" identified 12 randomized controlled trials that reported on changes in symptom 
severity for panic disorder. The control group was comprised of both usual care and waiting list 
participants. There was a statistically significant improvement in symptoms of panic disorder 
among participants who received iCBT compared to control groups (Hedges’ g = 1.31; 95% Cl 
0.85—1.76)). 


Dedert et al?? identified seven randomized controlled trials that reported on panic disorder. The 
control group was defined as waiting list, usual care, or attention/information control. Attention/ 
information controls received support or psychoeducation on the symptoms or disorder being 
targeted. A large SMD, indicative of a significant reduction in symptoms, was found in favour of 
participants who received iCBT compared to waiting list, usual care, or attention control (SMD = 
-1.08, 95% Cl -1.45 to -0.72).7 


Andrews et al?" identified nine randomized controlled trials that reported on improvement in 
symptoms for generalized anxiety disorder. Internet-delivered CBT demonstrated a statistically 
significant improvement in symptoms compared to usual care and waiting list (Hedge’s g = 0.70, 
95% Cl 0.39-1.01). 


Andrews et al*' also identified 11 randomized controlled trials that reported on changes in 
symptom severity for social anxiety disorder. Internet-delivered CBT demonstrated a statistically 
significant improvement in symptoms compared to usual care and waiting list (Hedge’s g = 0.92, 
95% Cl 0.76—1.08). 


Quality of Life 


Dedert et al?? identified three randomized controlled trials that reported on health-related quality 
of life for generalized anxiety disorder. A statistically significant standardized mean difference 
was found in favour of iCBT compared with control (SMD = 0.57, 95% Cl 0.27—-0.87). 7328 


Dedert et al?? identified six randomized controlled trials that reported on health-related quality of 
life for panic disorder. A statistically significant standardized mean difference was found in 
favour of iCBT compared to control (SMD = 0.49; 95% Cl 0.23—0.75). 


Patient Dropout and Treatment Adherence 


Dedert et al?° reported treatment adherence as the percentage of patients completing alll 
planned sessions and as the mean number of sessions completed stratified by the level of 
support. Dedert et al®° also identified four randomized controlled trials that reported on treatment 
adherence for panic disorder, and there was variation based on the level of support. Support 
provided alongside iCBT included feedback provided by a technician (nonlicensed staff) or 
Clinician (licensed professional) based on the participant’s previous interactions with the 
program, and psychoeducation. Live support comprised phone sessions, a scheduled chat on 
internet forums, or instant messaging with either technicians or clinicians. Eighty percent of 
participants completed sessions when there was live support, and 24% to 95% of participants 
completed sessions when there was delayed support. 
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Internet-Delivered CBT Compared to Face-to-Face Cognitive Behavioural Therapy 
Symptoms and Response to Treatment 


Dedert et al reported*? data on four randomized controlled trials on panic disorder that assessed 
changes in symptoms. There was variation in the face-to-face CBT ranging from ten 2-hour 
group sessions in one study and 6 to 12 individual sessions in three other studies. Three 
randomized controlled trials provided support (e.g., feedback on the participant’s previous 
interactions with iCBT and/or psychoeducation), and one trial used a mobile palmtop as an 
adjunct to face-to-face CBT. The authors found no statistically significant difference in 
symptoms of panic disorder for participants who had undergone iCBT compared with those who 
had undergone face-to-face CBT for three randomized controlled trials (SMD = 0.06, 

95% Cl -0.19 to 0.31). 22° Similarly, one randomized controlled trial that provided iCBT as an 
adjunct to face-to-face CBT found no difference in symptoms of panic disorder compared with 
face-to-face CBT (SMD = -0.42, 95% Cl -0.87 to 0.02). 


Arnberg et al?° found no difference in symptoms for panic disorder between individuals 
randomized to guided iCBT that included therapist (i.e., clinical) support when compared to 
group face-to-face CBT.”° Noninferiority was not established as the confidence interval included 
the predefined noninferiority margin of d = -0.20. Similarly, no difference was found in 
symptoms of panic disorder between individuals randomized to guided iCBT that included 
therapist (i.e., clinical) support compared with individual face-to-face CBT. This study was not 
designed to assess noninferiority. 


Arnberg et al?° identified one randomized controlled trial that reported on individuals with social 
phobia. There was a statistically significant difference in symptoms of social phobia for 
participants who received guided iCBT that included therapist (i.e., clinical) support when 
compared to group face-to-face CBT (d = 0.41, 95% Cl 0.03 to 0.78). 


Quality of Life 


Dedert et al’? identified three randomized controlled trials that reported on health-related quality 
of life for panic disorder, comparing iCBT to face-to-face CBT. There was no statistically 
significant difference in quality of life for participants who received iCBT compared to face-to- 
face CBT (SMD = -0.07, 95% Cl -0.34 to 0.21). 


Internet-Delivered CBT Compared to Active Control 
Symptoms and Response to Treatment 


Kampmann et al® reported on symptoms of social anxiety disorder, comparing iCBT with active 
control. The authors did not define the active control condition. 


Internet-delivered CBT demonstrated a small to medium statistically significant improvement in 
symptoms of social anxiety disorder compared to active control conditions post-assessment 
(Hedges’s g = 0.38, 95% Cl 0.13-0.62). Similarly, a small to medium statistically significant 
improvement in symptoms of social anxiety disorder occurred compared to active control 
conditions at 26 months follow-up (Hedges’s g = 0.23, 95% Cl 0.04—0.43).?° 
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Quality of Life 


Kampmann et al?° found a small statistically significant improvement in quality of life in favour of 
iCBT compared to active control conditions post-assessment (Hedges’g = 0.30, 95% Cl 0.10- 
0.50). 


Internet-Delivered CBT Compared to Passive Control 
Symptoms and Response to Treatment 


Kampmann et al*° reported on symptoms of social anxiety disorder, comparing iCBT with 
passive control. However, the authors did not define the passive control condition. The authors 
of the systematic review reported that, compared with passive control conditions, iCBT 
demonstrated a large statistically significant improvement in symptoms of social anxiety disorder 
at post-assessment (Hedges’s g = 0.84; 95% Cl 0.72—-0.97). An exploratory analysis of two 
studies did not show a statistically significant improvement in symptoms of social anxiety 
disorder for iCBT relative to passive control conditions until 5 months (Hedges’s g = 0.12; 

95% Cl -0.17 to 0.42).”° 


Quality of Life 


Kampmann et al® found a moderate statistically significant improvement in quality of life in 
favour of iCBT compared to passive control (Hedges’s g = 0.57; 95% Cl 0.2—0.93). 


See Table 3 for a summary of results for anxiety disorders. 
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Table 3: Summary of Results of Included Systematic Reviews for Anxiety Disorders 


Author, Year 


No., Type of 
Studies/ 
No. of Participants 


Results 


Quality Assessment 


Internet-delivered CBT compared with waiting list 


Symptoms and Response to Treatment 


Arnberg et al, 20147° 


Generalized anxiety 
disorder 


4 RCTs/132 


Guided iCBT vs. waiting list 
SMD = 0.84, 95% Cl 0.45-1.23 


GRADE: @@ low 


Adelman et al, 2014’ 


Generalized anxiety 


iCBT vs. waiting list 


disorder 
4 RCTs/317 


SMD = 1.06, 95% Cl 0.82—1.30 


Jadad score unclear for the 
subgroup of studies@ 


Dedert et al, 20137 


Generalized anxiety 


iCBT vs. waiting list 


disorder 


SMD = -0.94, 


Stregnth of evidence: moderate 


4 RCTs/321 95% Cl -1.34 to -0.54 

Arnberg et al, 20147° Panic disorder Guided iCBT vs. waiting list GRADE: @@ low 
4 RCTs/132 Small to very large effects 

Adelman et al, 20147 Panic disorder iCBT vs. waiting list Jadad score unclear for the 
8 RCTs/406 SMD = 1.15, 95% Cl 0.94-1.37 ue Se 

Arnberg et al, 2014?° Social phobia Guided iCBT vs. waiting list GRADE: ®@@ moderate 
8 RCTs/356 SMD = 0.85, 95% Cl 0.66—1.05 

Adelman et al, 20147 Social anxiety iCBT vs. waiting list Jadad score unclear for the 
disorder subgroup of studies* 


9 RCTs/not specified 


SMD = 0.91, 95% Cl 0.74—1.07 


Quality of Life 


Richards et al, 201574 


Generalized anxiety 
disorder 
2 RCTs/157 


SMD = 0.38, 95% Cl 0.08—0.67 


Risk of bias using the Cochrane 
Collaboration criteria was not 
reported for the subgroup of 
studies 


Patient Dropout/Adherence 


Dedert et al, 201378 


Generalized anxiety 
disorder 


2 RCTs/137 


Guided iCBT°: 75% completion 
Guided iCBT°: 11% completion 


Strength of evidence not reported 


Internet-delivered CBT compared with a combination of usual care, waiting list, and/or information control 


Symptoms and Response to Treatment 


Andrews et al, 20187! 


Panic disorder 
12 RCTs/584 


Social anxiety 
disorder 


11 RCTs/950 


Generalized anxiety 
disorder 
9 RCTs/1,103 


Panic disorder 
7 RCTS/333 


Hedges’s g = 1.31, 
95% Cl 0.85-1.8 

Hedges’s g = 0.92, 
95% Cl 0.76—1.08 


Hedges’s g = 0.70, 
95% Cl 0.39-1.0 


SMD = -1.08, 
95% Cl -1.45 to -0.72 


Low/unclear 


Low/unclear 


Low 


Strength of evidence: moderate 


Quality of Life 


Dedert et al, 201378 


Generalized anxiety 
disorder 
3 RCTs/176 


Panic disorder 
6 RCTs/250 


SMD = 0.57, 95% Cl 0.27-0.87 


SMD = 0.49, 95% Cl 0.23-0.75 


Strength of evidence: low 


Strength of evidence: moderate 
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Author, Year No. of Participants 


Results 


February 2019 


Quality Assessment 


Patient Dropout and Treatment Adherence 


Dedert et al, 20137° Panic disorder 


4 RCTs/313 


80% of participants completed 
sessions when there was live 
support; 24-90% completed 
sessions when there was delayed 
support 


Strength of evidence: not reported 


Internet-delivered CBT compared with face-to-face CBT 


Symptoms and Response to Treatment 


Arnberg et al, 20147° Specific phobia 


1 RCT/30 


Social phobia 
1 RCT/126 


Guided iCBT vs. brief therapist-led 
exposure 


No change (no effect size reported) 


Guided iCBT vs. group face-to-face 
CBT 


d = 0.41, 95% Cl 0.03-0.78 


GRADE @© very low 


GRADE: @@ low 


Dedert et al, 20137° Panic disorder 


1 RCT/49 


Panic disorder 
1 RCT/113 


Panic disorder 
3 RCTs/248 


1 RCT/121 


Guided iCBT vs. individual face-to- 
face CBT 


Not statistically significant 


Guided iCBT vs. group face-to-face 
CBT 


d = 0.00 (95% Cl -0.41 to 0.41) 


iCBT vs. face-to-face CBT 
SMD = 0.06, 95% Cl -0.19 to 0.31 


SMD = -0.42, 95% Cl -0.87 to 
0.02° 


GRADE: © very low 


GRADE: © Very low 


Strength of evidence: moderate 


Quality of Life 


Dedert et al, 201378 Panic disorder 


3 RCTs/239 


SMD = -0.07, 95% Cl -0.34 to 0.21 


Strength of evidence: moderate 


Internet-delivered CBT compared with active control 


Symptoms and Response to Treatment 


Kampmann et al, 20167 Social anxiety 


disorder 

Number of studies 
not specified/number 
of participants not 
specified 


A small to medium effect was found 
when iCBT was compared with 
active control conditions at post- 
assessment (Hedges’s g = 0.38, 
95% Cl 0.13-0.62, SE = 0.13, 

P <0.01, k = 8) and at follow-up 2° 
(Hedges’s g = 0.23, 95% Cl 0.04— 
0.43, SE = 0.10, P= 0.02; k = 5) 


Risk of bias using the Cochrane 
Collaboration criteria was not 
reported for the subgroup of 
studies 


Quality of Life 


Social anxiety 
disorder 


Number of studies 
not specified/number 
of participants not 
specified 


Kampmann et al, 20167 


Hedges’s g = 0.30, 95% Cl 0.10— 
0.50, SE = 0.10, P< 0.01,k=3 


Risk of bias using the Cochrane 
Collaboration criteria was not 
reported for the subgroup of 
studies 
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Results 


February 2019 


Quality Assessment 


Internet-delivered CBT compared to passive control 


Symptoms and Response to Treatment 


Kampmann et al, 20167 


Social anxiety 
disorder 

Number of studies 
not specified/number 
of participants not 
specified 


Hedges’s g = 0.84, 95% Cl 0.72- 
0.97, SE = 0.07, P< 0.001, k = 16 


Exploratory analysis including only 
two studies of iCBT relative to 
passive control conditions at 
follow-up 1‘ (Hedges’s g = 0.12, 
95% Cl -0.17 to 0.42, SE = 0.15, 


Risk of bias using the Cochrane 
Collaboration criteria was not 
reported for the subgroup of 
studies 


P=0.412, k = 2) 
No., Type of 
Studies/ 
Author, Year No. of Participants Results Quality Assessment 


Quality of Life 


Kampmann et al, 20167 


Social anxiety 
disorder 

Number of studies 
not specified/number 
of participants not 
specified 


A medium effect for iCBT 
compared to passive control 
(Hedges’s g = 0.57, 95% Cl 0.21- 
0.93, SE = 0.31, P< 0.01, k = 2) 


Risk of bias using the Cochrane 
Collaboration criteria was not 
reported for the subgroup of 
studies 


Abbreviations: Cl, confidence interval; iCBT, internet-delivered cognitive behavioural therapy; SE, standard error; SMD, standardized mean difference. 
@The authors of the health technology assessment were unable to extract the Jadad score for the subgroup of studies. 
Real-time interactions with study technicians (nonlicensed staff) or clinicians (licensed professionals), including phone sessions, a scheduled chat on 
internet forums, or instant messaging. 
°Real-time communication with technician (nonlicensed staff) or clinician (licensed professionals) was delayed. 
4Internet-delivered CBT as an adjunct vs. face-to-face CBT. 
°Follow-up period was defined as 6 months and greater. 
‘Follow-up period was defined as less than 5 months. 


Adverse Events 


Dedert et al?° planned to assess outcomes of safety of iCBT across disorders (e.g., emergency 
department visits, hospital admissions related to the disorder being treated, and self-harm 
behaviors); however, the data were unavailable from the included studies and were not reported 
by the systematic review authors. No other systematic reviews included in this assessment 
reported any data on safety. 


Discussion 


Internet-delivered CBT is one type of psychotherapy for the treatment of mild to moderate major 
depression and anxiety disorders. Of the seven systematic reviews included in this clinical 
evidence review, we found little evidence on guided iCBT. None of the systematic reviews 
presented data on unguided iCBT. One systematic review evaluated the noninferiority of iCBT 
compared to group face-to-face CBT and individual face-to-face CBT for anxiety disorders.”° 
Overall, the level of support (guided or unguided) associated with iCBT in the included 
systematic reviews was limited. 


There is moderate-quality evidence suggesting that, compared with waiting list, guided iCBT is 
effective and likely results in symptom improvement for mild to moderate major depression and 
social phobia. Compared with waiting list, guided iCBT may also improve the symptoms of 
generalized anxiety disorder and panic disorder. However, there is uncertainty about the 
effectiveness of iCBT compared with individual or group face-to-face CBT. 
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Although there was a statistically significant improvement in quality of life for individuals with 
generalized anxiety disorder who received iCBT compared to waiting list, these results should 
be interpreted with caution as the lower limit of the 95% confidence interval (i.e., 0.08) is very 
close to the zero cut-off. While iCBT demonstrated an improvement in quality of life for people 
with generalized anxiety disorder or panic disorder compared to a combination of usual care, 
waiting list, and information control, the strength of evidence is low to moderate. 


Guided iCBT may provide a promising option for the treatment of mild to moderate major 
depression and select anxiety disorders. Moreover, iCBT may expand treatment for individuls 
unable or unwilling to access face-to-face CBT. 


Limitations 


The characteristics of the study populations and the number of randomized controlled trials for 
each subgroup analysis varied in the included systematic reviews. Characteristics of the study 
participants included a high level of educational attainment and employment, and recruitment 
was via the internet or advertisements.2° The potential selection bias identified limits the 
generalizability of these findings. Furthermore, results from single-trial efficacy studies 
conducted on guided iCBT compared to group face-to-face CBT and individual face-to-face CBT 
for panic disorder, and guided iCBT compared to brief therapist-led exposure for specific 
phobias have limited external validity due to the small sample sizes.?° 


We undertook an overview of systematic reviews. There may be randomized controlled trials 
published since then that we have not captured. A rapid review by CADTH provides an 
assessment of the randomized controlled trials published since the systematic reviews 
assessed in this report.?” 


Conclusions 


Compared with usual care, iCBT: 
e Significantly improves symptoms for mild to moderate major depression (GRADE not 
conducted) 


Compared with waiting list, guided iCBT: 
e Significantly improves symptoms of mild to moderate major depression (GRADE 
moderate) 
e Significantly improves symptoms of generalized anxiety disorder (GRADE low) 
e Significantly improves symptoms of panic disorder (GRADE low) 
e Significantly improves symptoms of social phobia (GRADE moderate) 


Compared with face-to-face CBT, guided iCBT: 
e Did not significantly improve symptoms of panic disorder (GRADE very low) 
e Did not significantly improve symptoms of social phobia (GRADE low) 
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ECONOMIC EVIDENCE 


Research Questions 


e What is the cost-effectiveness of unguided or guided iCBT compared with face-to- 
face CBT or usual care in the management of adults with mild to moderate major 
depression? 


e What is the cost-effectiveness of unguided or guided iCBT compared with face-to- 
face CBT or usual care in the management of adults with anxiety disorders? 


Methods 
Economic Literature Search 


We performed an economic literature search on February 21, 2018, for studies published from 
January 1, 2000, to the search date. To retrieve relevant studies, we developed a search using 
the clinical search strategy with an economic filter applied. 


We created database auto-alerts in MEDLINE, Embase, PsycINFO, and CINAHL and monitored 
them for the duration of the health technology assessment. We performed a targeted grey 
literature search of health technology assessment agency websites, PROSPERO, and Tufts 
Cost-Effectiveness Analysis Registry. See the Clinical Evidence literature search, above, for 
further details on methods used. See Appendix 1 for literature search strategies, including all 
search terms. 


Literature Screening 


A single reviewer screened titles and abstracts, and, for those studies likely to meet the 
eligibility inclusion criteria, we obtained full-text articles and performed further assessment for 
eligibility. 


Inclusion Criteria 


e English-language, full text articles published between January 1, 2000, and the search 
date 


e Individual-level economic evaluations conducted alongside randomized controlled trials 
(i.e., trial-based) or economic analyses based on decision analytic models (i.e., model- 
based) 

e Studies in adults with major depression or an anxiety disorder that falls within the DSM-5 
criteria 

e Studies comparing unguided or guided iCBT (by a therapist or a coach) with face-to-face 
CBT, pharmacologic therapies, treatment as usual, or no treatment (e.g., waiting list or 
placebo) 


Exclusion Criteria 


e Narrative reviews of the literature, study protocols, guidelines, conference abstracts, 
commentaries, letters, and editorials 

e Economic evaluations examining iCBT used solely for the treatment of postnatal 
depression or depression or anxiety co-occurring with chronic conditions 
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(e.g., depression coexisting with chronic diseases such as cardiovascular disease, 
dementia, diabetes mellitus, or inflammatory bowel disease) 

e Economic evaluations examining the value of non-traditional CBT (e.g., mindfulness 
CBT), CBT delivered via bibliotherapy, or CBT described as computerized CBT for which 
no further analysis was done specifically for iCBT 

e Feasibility studies exploring different models of care for the treatment of major 
depression or anxiety disorders that do not report economic outcomes 


e Noncomparative studies reporting the costs of iCBT or cost-of-illness studies 


Outcomes of Interest 


e Incremental costs 

e Incremental effectiveness outcomes (e.g., quality-adjusted life-years [QALYs], disability- 
adjusted life-years [DALYs]) 

e Incremental cost-effectiveness ratio (ICER) 

e Incremental net benefit (INB) 


Data Extraction 


We extracted relevant data on the following: 


e Publication source (i.e., author's name, location, publication year) 

e Study design 

e Study population 

e Interventions and comparators 

e Outcomes (e.g., health outcomes, costs, and incremental cost-effectiveness ratio) 


Study Applicability and Limitations 


We determined the usefulness of each identified study for decision-making by applying a 
modified quality appraisal checklist for economic evaluations that was originally developed by 
the National Institute for Health and Care Excellence (NICE) in the United Kingdom to inform 
development of NICE’s clinical guidelines. We modified the wording of the questions to remove 
references to guidelines and to make it Ontario-specific. Next, we separated the checklist into 
two sections. In the first section, we assessed the applicability of each study to the research 
question (directly, partially, or not applicable). A summary is presented in Appendix 4. In the 
second section, we assessed the limitations (minor, potentially serious, or very serious) of the 
studies that we found to be directly or partially applicable. 
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Results 


Literature Search 
The literature search yielded 504 citations published from January 1, 2000, to February 21, 


2018, after removing duplicates. We excluded a total of 456 articles based on information in the 


title and abstract and obtained 48 potentially relevant articles for further assessment. Forty 
studies met the inclusion criteria and were assessed to establish the applicability of their 
findings to the Ontario context. Figure 2 presents the PRISMA diagram for the economic 
evidence review. 


Records identified through Additional records identified 
database searching (n = 1,078) through other sources (n = 13) 


Identification 


Records after duplicates removed 
(n = 504) 


Records screened Records excluded 
(n = 504) (n = 456) 


Full-text articles assessed for Full-text articles excluded (n = 8): 
eligibility (n = 48) ° Duplicate studies (n = 2) 
e Not intervention of interest (n = 2) 
e Prevention/implementation (n = 2) 
e Feasibility or costing (n = 2) 


Eligibility 


Studies included in qualitative 
synthesis (n = 40): 
Systematic reviews (n = 9) 
Original studies, major depression 
(n = 15) 
Original studies, anxiety (n = 16) 


Studies included in quantitative 
synthesis (n = 0) 


Figure 2: PRISMA Flow Diagram—Economic Search Strategy 


Source: Adapted from Moher et al.?6 
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Review of Included Economic Studies 


Of the 40 eligible studies, 9 were systematic literature reviews?°?®*> that examined the cost- 
effectiveness of iCBT among other psychological therapies for either major depression or 
anxiety disorders, and 31 were original research studies focusing on the cost-effectiveness of 
iCBT in adults with major depression (n = 15)%**° or in adults with an anxiety disorder 

(n = 16).9:50-64 


Systematic Reviews: The Cost-Effectiveness of iCBT for the Management of 
Major Depression or Anxiety Disorders 


We identified nine relevant systematic reviews including data on the potential cost-effectiveness 
of iCBT published between 2012 and 2018.7°:8°5 All reviews examined other psychological 
treatments in addition to iCBT: four reviews were focused on major depression,”°*' two 
examined anxiety disorders,**°° and three reviews examined internet interventions for mental 
health conditions in general or for mood and anxiety disorders altogether.2°***° The reviews 
examining depression or anxiety disorders included mixed study populations and types of 
psychological treatments in addition to guided and unguided iCBT, which were examined ina 
wide range of included studies. In general, results were mixed. Most reviews suggested that 
iCBT could represent an economically viable treatment alternative over control.*° The one 
exception, Kolovos et al,°° included individual-level participant data and conducted a meta- 
analysis and concluded that guided iCBT was not a cost-effective option compared with control. 
None of the studies described in detail the differences between unguided and guided iCBT, and 
there was a great variability of cost-effectiveness estimates. We performed our own systematic 
review of the literature to inform our primary economic evaluation and to gain a better 
understanding of relevant values for our model input parameters, such as changes in health- 
related quality of life weights associated with progression of depression and anxiety and with 
iCBT treatment. 


We reviewed all original studies relevant to our research questions. We then compared their 
study designs, including perspective, the time horizon of analysis, study populations, and 
comparative strategies. Lastly, we summarized the cost-effectiveness findings. Tables 4 and 5 
summarize the characteristics and results of the included original studies. 


Original Studies: The Cost-Effectiveness of iCBT for the Management of Mild to 
Moderate Major Depression 


Study Design 


Of the 15 included studies (see Table 4), 12 were individual-level cost-effectiveness analyses 
conducted alongside clinical trials. Six were from the United Kingdom**“? (the report by 
Littlewood et al®* was later published by Duarte et al°’; it was counted as the same publication in 
Table 4), one study was from Spain,*? one from Australia, and three were from the 
Netherlands.***” Three studies were model-based cost-effectiveness analyses (two from 
Australia*®*° and one from the United Kingdom*’). 


Study Population 


All economic evaluations were done with adult participants who had major depression, and the 
majority included mild to moderate disease severity in the inclusion criteria. Prior episodes of 
major depression and use of medications at baseline was reported by participants in four 
studies.°*4°? One study was specifically conducted in older adults aged 60+ years,** and another 
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study*> targeted employed individuals (mean age: 43 years).*° The remaining studies recruited 
participants from hospitals or the general population. All but one study in older Australian 
adults** included relatively large samples of about 300 to 700 participants, with the mean age 
ranging from 34 to 50 years, and with females composing between 50% and 80% of the study 
populations. 


Analysis Perspective 


The analysis perspective varied between the studies and may have been affected by the 
features of each country’s health care system. Most studies conducted in the United Kingdom 
were done from a health sector perspective. Most of the studies done in the Netherlands, 
Australia, and Spain were done from a societal perspective. 


Time Horizon 


The duration of follow-up in trial-based cost-effectiveness analyses or of the time horizon in 
model-based cost-effectiveness analyses was short, ranging from 8 weeks to 12 months in most 
studies. This indicates a short-term, non-repetitive use of iCBT. Only two cost-utility analyses 
(one trial-based,**°” and another model-based*’) examined the benefits and costs of 6- to 
8-session courses of unguided or coach-guided iCBT over a period of 18 months. 


Interventions and Comparators 


Unguided iCBT. Six studies used unguided iCBT as an intervention strategy. The unguided 
iCBT consists of 4 to 8 sessions, along with homework assignments. In most studies, the 
sessions were designed to be completed weekly. In three individual-level cost-effectiveness 
studies, usual care was compared with unguided or minimally guided iCBT (where minimal 
guidance was defined as technical guidance on use of the software).°69”41:43:46 One study by 
Gerhards et al*® compared iCBT alone to usual care alone and to iCBT combined with usual 
care. Another study by McCrone et al*? compared iCBT combined with usual care to usual care 
alone. Only one model-based study by Solomon et al*® examined the benefits and costs of 
unguided iCBT to usual care and to face-to-face CBT.” 


Guided iCBT. In total, nine studies examined therapist-guided iCBT.°°4049-5.47-4950 Durations 
varied from 6 to 16 sessions completed on a weekly basis, along with homework assignments. 
In four studies,*°°9475° including three from the United Kingdom, iCBT was guided by a coach, a 
non-regulated mental health worker trained to provide low-intensity CBT. Highly trained 
regulated therapists (e.g., clinicians or psychologists) were employed in five studies done in 
Spain, *° the Netherlands,*° Australia***? and the UK *°. In seven studies, guided iCBT was 
compared with usual care only. One study*” compared guided iCBT with problem-solving 
therapy; 4” another®® compared guided iCBT supported by a coach (i.e., a non-regulated mental 
health worker) with unguided iCBT.*® 


Control Treatment. |n general, usual care consisted of treatment provided by a general 
practitioner and included active surveillance, medications or counselling, and psychoeducation 
according to country-specific clinical guidelines. Another frequently used control option was 
waiting list. One study by Phillips et al*! that examined unguided iCBT used a self-help mental 
health website as a control comparator.*! 
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Cost-Effectiveness Results 


In 11 of the 14 included studies, ICER estimates associated with the incremental cost- 
effectiveness of unguided or guided iCBT were below country-specific willingness-to-pay 
thresholds, indicating that these types of CBT could be economically attractive options in the 
management of mild to moderate depression. However, results should be interpreted with 
caution as there were some uncertainties around the ICER in the probabilistic sensitivity 
analyses. Depending on the duration and type of cost—utility analysis, the probability of 
unguided or guided iCBT being cost-effective ranged from 52% to more than 95% (at country- 
specific willingness to pay thresholds). Moreover, the probability of cost-effectiveness of guided 
iCBT versus unguided iCBT was 55% at a willingness to pay of £30,000/QALY.*® 


In most studies, the costs associated with guided iCBT were greater than the cost of usual care; 
however, the incremental benefits (expressed in QALYs) remained uncertain. For instance, in 
several studies, unguided or guided iCBT alone was associated with increments of 0.01 to 

0.02 QALYs. Comparatively, slightly larger increases of 0.03 to 0.04 QALYs were shown in two 
studies that examined the cost-effectiveness of guided or unguided iCBT when combined with 
usual care.*°-44 Exceptionally high increases in QALYs were also determined in a modeling 
study by Kaltenthaler et al®° and an individual-level cost—utility analysis by Romero-Sanchiz.*? In 
these studies, increments in QALYs were shown to be about 0.08 for guided or unguided iCBT 
compared with usual care. Lastly, a study by Brabyn et al®* that compared guided to unguided 
iCBT found a small gain in QALYs of 0.003 with guided iCBT. 


Three cost-utility analyses suggested that unguided or guided iCBT is probably economically 
unattractive compared with usual care for the management of mild to moderate 
depression*®*’3945 (see Table 5). A cost—utility analysis alongside a large randomized controlled 
trial of 691 participants with prevalent or new major depression*®*” found that two commercially 
available unguided iCBT software programs (“MoodGYM” and “Beating the Blues”), which 
included 6 to 8 online sessions with homework assignments, were no better than usual care 
after 18 months of follow-up.**°” Another cost-—utility analysis alongside a large randomized 
controlled trial of 609 participants with mild to moderate major depression®? found that coach- 
guided iCBT software (“Living Life to the Full”) was not cost-effective compared with usual care 
after 12 months of follow-up.*° After accounting for a large amount of missing data, the reported 
ICER was £132,630/QALY, and the probability of cost-effectiveness of iCBT versus usual care 
at a willingness-to-pay threshold of £20,000/QALY was 30%.°*? In contrast, the results were 
economically attractive in the complete case analysis (ICER: £3,850/QALY), but the authors 
indicated that the results of this complete-case analysis should be considered biased. 


Gerhards et al *° examined the cost-effectiveness of guided iCBT (software: “Happy@Work’) in 
a convenience sample of 231 employed adults with mild to moderate depression and found no 
incremental benefit of guided iCBT versus usual care. With the mean difference in QALYs 
estimated at -0.001 (95% Cl -0.04 to 0.04), the authors determined that iCBT did not seem to 
be economically attractive despite some cost savings. Overall, the study estimated an ICER of 
€533,000/QALY lost—a savings of over €530,000 per QALY lost (Table 4). Finally, in a cost— 
utility modeling study by Solomon et al*®, unguided iCBT (software: “My Compass”) was cost- 
saving compared with usual care; however, despite decreases in the mean costs of about 
$2,000, due to a decrement in the mean QALYs of 0.03, unguided iCBT was found to be less 
cost-effective than face-to-face CBT at the commonly used willingness-to-pay threshold of 
$50,000 AUD/QALY in Australia. An ICER of an estimated $66,000 AUD/QALY indicated a 
savings of $66,000 AUD per QALY lost. 
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Original Studies: The Cost-Effectiveness of iCBT for the Management of 
Anxiety Disorders 


Study Design 


Of the 16 included studies (see Table 5), 11 were individual-level cost-effectiveness analyses 
conducted alongside randomized controlled trials (seven from Sweden,**°**' two from 
Australia,®*°* one from the United States,°° and one from the United Kingdom*®*), and five were 
model-based cost-effectiveness analyses (one from the United States,® three from the United 
Kingdom,?°°* and one from Australia®'). Some studies were performed by the same research 
groups, presenting the short-term and long-term results in separate publications. For example, 
Hedman at al (2011),°° Hedman at al (2014),°%5° and El Alaoui et al (2017)°' reported 6-month 
to 4-year follow-up results for social anxiety disorder. Hedman et al (2013)°” and Hedman et al 
(2016)°”®° reported 12-week and 1-year follow-up results for health anxiety disorder. 


Study Populations 


A majority of the economic evaluations we found focused on adults with panic disorder 

(n = 5)%°°52:54 and social anxiety disorder (n = 5),°%:575*5'.§4 followed by generalized anxiety 
disorder (n = 2),®*®° severe health anxiety (n = 2),5”® and other anxiety (n = 2).°>5° All economic 
evaluations alongside randomized controlled trials that examined iCBT as a single intervention 
had small to moderately large samples of 72 and 193 participants. The exception was a study 
by Joesch et al°® examining the cost-effectiveness of therapist-guided iCBT for any anxiety 
disorder within a collaborative care program, which included a sample of 690 participants. In 
many of the studies, participants were between 34 and 49 years of age and were predominantly 
female (> 70%). Almost all study participants had a long-standing anxiety disorder, used 
medications, and/or had already been treated with a psychological therapy. 


Analysis Perspective 


The analytic perspective varied between studies. Most studies conducted in Sweden adopted a 
societal perspective, while studies conducted in the United Kingdom, Australia, and the United 
States adopted a health sector perspective. 


Time Horizon 


The duration of follow-up depended on the type of anxiety disorders even though the course of 
an iCBT program was short and occurred only one time. Studies of adults with panic disorder 
lasted from 6 weeks to 12 months. Studies of adults with social phobia lasted for 6 months to 5 
years. Studies of middle-aged adults with severe health anxiety, any anxiety, and of older adults 
with GAD lasted for 8 weeks to 18 months. Only one modeling study® examined the lifetime 
cost-effectiveness of iCBT in adults with GAD. Kumar et al®* conducted a Markov cohort 
decision analytic study assuming continuous benefits of iCBT over 5 years based on the 
estimates from a 3-month pilot coach-based iCBT program (software: “Lantern’). 


Interventions and Comparators 


Unguided iCBT. Only three studies, two model-based°°* and one trial-based,°? examined the 
cost-effectiveness of unguided iCBT versus control in adults with social anxiety or panic 
disorders. The duration of iCBT course was between 6 and 12 weeks. 
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Guided iCBT. The remaining 13 studies examined the cost-effectiveness of guided iCBT over 
the control. One of these studies was embedded within a collaborative care program that also 
included case management and coordination of psychiatric care.*> Overall, the iCBT programs 
consisted of 5 to 15 modules with homework assignments that were taken over a period of 8 to 
15 weeks. 


Control. Benefits and costs of iCBT programs were compared with treatment as usual, including 
medications, waiting list control, online relaxation/stress management programs or discussion 
forums, and face-to-face CBT (group or individual format). 


Cost-Effectiveness Results 


Most studies showed that guided or unguided iCBT for the treatment of anxiety disorders 
represented good value for money (i.e., cost-saving or cost-effective at a country-specific 
willingness-to-pay threshold) when compared with control. When compared with usual care, 
guided iCBT was associated with QALY increments from 0.01 to 0.05. Guided iCBT for social 
phobia had equal benefits to group CBT in noninferiority trials by Hedman et al.9”°° A 12-month 
modeling study by Kaltenthaler et al®° indicated a QALY increment of 0.06 for unguided iCBT 
(software: “FearFighter”) versus a computerized relaxation program, and a decrease in QALY of 
0.01 for unguided iCBT versus individual face-to-face CBT. In many of the studies, the 
probability of cost-effectiveness of iCBT versus control was high and ranged from 70% to 

over 95%. 
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Table 4: Results of Economic Literature Review—Cost-Effectiveness of iCBT for the Treatment of Major Depression 


Methods Results 

Name, 
Year, Study Design and Interventions / 
Location Perspective Population Comparators Health Outcomes Costs Cost-Effectiveness 
Duarte et al, Individual-level cost— 218 yr with prevalent or e Interventions: (1) e —_ Beating the Blues, Currency: £ Base-case analysis: 
2017°; utility analysis new major depression Beating the Blues, a MoodGYM, and Year: 2011-2012 Neither iCBT program 
Littlewood et : : (PHQ-9 score of 210) not commercial pay-to-use control: : with minimal technical 

P I Beating the BI 
al, 2015, a oe biel using psychotherapy at iCBT program, plus Total QALYs ceria ai guidance (Beating the 

: open, three-arm, parallel ; usual care 

United RCT with simple baseline (N = 691) usual GP care (n = (measured by EQ-5D): Total costs (mean): Blues or MoodGYM 
Kingdom 210), or (2) MoodGYM, 1.33 (SD 0.037), 1.36 plus usual GP care) 


randomisation; REEACT 
UK NHS 

Time horizon: 18 months 
Discount rate: 3.5% 


Beating the Blues, 
MoodGYM, control 
Mean age: 39.61 
(SD12.34), 39.43 (SD 
12.96), and 40.52 (SD 
12.64), respectively 


Beating the Blues vs 
MoodGYM vs control 
Female participants: 
67.62%, 64.88%, and 
68.20%, respectively 


Most participants with 
moderate depression: 
PHQ-9 = 17; prior major 
depression: 68% to 75%; 
prior antidepressant use: 
85% to 89%; prior 
anxiety: 79% to 81% 


a free-to-use iCBT 
program, plus usual GP 
care (n = 242) 


Comparator: usual GP 
care (n = 239) 


Unguided and low- 
intensity guided iCBT: 
(1) Beating the Blues, a 
15-min introductory 
video followed by eight 
50-min therapy 
sessions, with 
homework exercises 
between sessions, or 2) 
MoodGYM, 6 
interactive 
modules/sessions (30— 
45 min each) 

Both groups had low- 
intensity (technical) 
weekly phone 
support/calls to engage 
and troubleshoot the 
programs (not therapy) 


GP care according to 
NICE guidelines, 
stepped care, use of 
medications, or other 
psychotherapies if 
necessary (6% to 7% of 
participants took 
medications) 


(SD 0.03), and 1.39 
(SD 0.03), respectively 


Beating the Blues vs 
usual care: 

Adjusted mean 
difference: -0.043 
(95% Cl -0.12 to 0.03) 


Intervention vs usual 
care 

Mean QALYs at 18 
mo: 1.0 (SD 0.3) vs 1.0 
(SD 0.3) 


MoodGYM vs usual 
care 

Adjusted mean 
difference: -0.015 
(95% Cl -0.09 to 0.06) 


104.2 (95% Cl -67 to 
276) 


MoodGYM vs usual 
care 

Mean difference: 
-106.1 (95% Cl -262 
to 49) 


Beating the Blues 
Software licensing cost 
estimated at £50 per 
user 

MoodGYM 

Software licensing cost 
estimated at £0.66 per 
user 


The cost of iCBT 
included costs 
associated with the 
number and duration of 
phone calls (£21 unit 
cost per hour) 


was cost-effective 
(dominant) compared 
with usual care alone 


Probabilities of cost- 
effectiveness at wtp 
£20,000 per QALY: 
54% usual GP care, 
42% MoodGYM, and 
4% Beating the Blues 


Sensitivity analysis of 3 
scenarios: 

Scenario 1, SF-6D— 
source for QALYs: 
MoodGYM dominated 
usual GP care; 
probabilities of cost- 
effectiveness at wtp 
£20,000 per QALY: 
24% usual GP care, 
75% MoodGYM, and 
1% Beating the Blues; 
however, increments in 
QALYs for MoodGYM 
vs usual GP care not 
statistically significant 
mean: (0.006, 95% Cl 
-0.03 to 0.05) 
Scenario 2: Inclusion of 
depression-treatment 
costs only: usual GP 
care remains the 
dominant strategy 
Scenario 3: complete 
case analysis: usual 
GP care remains the 
dominant strategy 
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Methods Results 

Name, 

Year, Study Design and Interventions / 

Location Perspective Population Comparators Health Outcomes Costs Cost-Effectiveness 
Romero- Individual-level cost— 18-65 yr with a mild to Intervention: Smiling is Total QALYs (mean Currency: € Base-case analysis: 
Sanchiz et utility and cost- moderate major Fun per person, by EQ-5D- Year: 2014 Guided /unguided iCBT 
al, 2017, effectiveness depression who had not Unguided iCBT " (n = 3L) Total costs (mean) vs control, ICER: 

Spain analyses attended any 98), low-intensity At baseline (P = .435 guided iCBT/unguided 497/5,160 per QALY 
A pragmatic, psychological treatment therapist-guided iCBT Guided iCBT: 0.6794 iCBT/control, baseline Probability of iCBT 


multicentre (30 
primary care centers), 
open, three-arm, 
parallel RCT 


Societal perspective 


Time horizon: 12 
months 


Discount rate: 0% 


in the past year (N = 296) 
Mean age: 43 yrs 
Females: 72-76% 


(n = 96) 

Comparator: 

Improved TAU by a GP 
(n = 102) (GPs 
received 3-hr training in 
how to treat depression 
based on the NICE 
recommendations), 
including medications 


iCBT: 10 modules at 
1/wk 


Unguided iCBT 
included technical help 
without therapist 
support 


Guided iCBT included 
trained 
psychotherapist, help 
over email 


(SD +0.18) 
Unguided iCBT: 
0.6989 (SD +0.14) 
Control: 0.7076(SD 
+0.1482) 

At 12 mo (P= 0.13) 
Guided iCBT: 0.7883 
(SD +0.21) 
Unguided iCBT: 
0.7852 (SD +0.24) 
Control: 0.7059(SD 
+0.22): 

Mean difference 
Guided iCBT vs 
control: 0.0824 
Unguided iCBT vs 
control: 0.0824/0.0793 


vs 12 mo: 2,627 (SD 
+3,199)/3,119 (SD 
+3,125)/2,892 (SD 
+2,890), P = 0.537 vs 
1,757 (SD 
+3,636)/1,308 (SD 
+2,219)/1,716 (SD 
+2,437), P= 0.467 
Mean difference 
Guided iCBT/unguided 
iCBT vs control: 
40.9/-409.2 


being cost-effective 
compared with control 
for a wtp of 21,000 to 
25,000 per QALY: 
high, exact values NR 
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Methods Results 
Name, 
Year, Study Design and Interventions / 
Location Perspective Population Comparators Health Outcomes Costs Cost-Effectiveness 
Lee et al, Cost-utility analysis 18-60 yrs with mild to Intervention: guided Total QALYs (mean): Currency, Australian $ Base-case analysis: 
2017, Model-Based moderate major iCBT through Online iCBT vs control: 0.816 year: 2014 ICER: cost saving, 
Australia depression or anxiety MindSpot Clinic, "Well- vs. 0.798 dominant 


economic evaluation 


Health care payer 
perspective 


Time horizon: 12 
months 


Discount rate: 0% 


Mean Age: 37-39 yr 
Females: 77% 


being Course" 


Comparators: usual 
care, waitlist control 


4—6-session course 
delivered over 8 wk 
with homework 
assignments and 


Mean difference: 0.018 


Total costs (mean), 
iCBT vs control: 

(1) Excluding 
hospitalization costs: 
472.75 vs 488.04 

(2) Including 
hospitalization costs: 
472.75 vs 849.57 


Probability of iCBT 
being cost-effective: 
NR 


One-way deterministic 
analyses and 
scenarios: the ICER 
remained cost-effective 


weekly support from Mean difference: (<10,000/QALY 
clinicians plus (1) Excluding gained) 
reminders byautomated hospitalization costs: 
emailand phone 15.29 
Clinic data (costs and ey neiuelng . 
utilities as measured by hospitalization costs: 
EQ-5D) were used in 376.82 
the decision-tree model Total annual cost of 
attending Mindspot 
clinic: 391.60 
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Methods Results 

Name, 
Year, Study Design and Interventions / 
Location Perspective Population Comparators Health Outcomes Costs Cost-Effectiveness 
Brabyn et al, e _Individual-level cost— 218 yrs with moderate e Intervention: guided Total QALYs Currency: £ Base-case analysis, 
2016,°8 utility analysis major depression (PHQ-9 iCBT: weekly (measured by EQ-5D), Year: 2012-2013 including multiple 
United : score of 210) no supportive/ facilitative guided vs unguided Total : imputations: 
Kingdom A pragmatic, psychotherapy at phone calls plus iCBT at 12 mo: 0.700 ofa costs pan: All costs: guided iCBT 

multicentre, open, : Guided iCBT: 1,172 : 

two-arm, parallel baseline (N = 369) MoodGYM (n = 242) (SD +0.016) vs. 0.686 (SD +186.5) was dominant 

RCT; REEACT-2 Mean age: * Comparator: EUS) Unguided iCBT: 1,763 Biioe Medieaeo ele: 


e NHS and PSS 
perspective 


e Time horizon: 12 mo 
e Discount rate: 0% 


Guided iCBT: 41.0 (SD 
+13.8) 

Unguided iCBT: 40.3 (SD 
+3.7) 


Female: 
Guided iCBT: 67% 
Unguided iCBT: 62% 


Most patients with 
moderate depression: 
PHQ-9 score of 17 

Prior major depression: 
70% to 72% 

Prior antidepressant use: 
39% 


MoodGYM, unguided, 
minimally supported 
(technical only) (n = 
239) 


MoodGYM: 6 interactive 
modules/ 

sessions to be 
completed weekly (30— 
45 min each), plus 
homework. First 5 
interactive modules 
released sequentially, 
with a 6th session for 
consolidation and 
revision 

Additional support: 
phone facilitation and 
guidance, manualized 
/scripted, 10 and 20 min 
weekly to engage and 
motivate participants 

8 phone calls over 12— 
14 wk, between first 
contact and 4-mo 
follow-up. 

Trained non-regulated 
phone support workers 
are do not provide any 
CBT 


10-member early 
psychosis team 
consistingpsychiatrists, 
psychologists, 
occupational therapists, 
nurses, and health care 
assistants specifically 
trained in early 
psychosis 


Adjusted mean 
difference, guided vs 
unguided iCBT: 0.0026 


(438.8) 


Mean difference, 
guided vs unguided 
iCBT: 

All costs: -3.42 
Direct medical costs: 
CS hei7/ 


Non-medical costs of 
guided iCBT per 
patient: 42 


3,596/QALY 


Probability of cost- 
effectiveness of guided 
vs unguided iCBT at 
wtp 20,000 or 30,000 
per QALY was 55% 


Missing data at 12 mo: 
59% 

All analyses included 
multiple imputations 
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Name, 
Year, Study Design and Interventions / 
Location Perspective Population Comparators Health Outcomes Costs Cost-Effectiveness 
Dixon et al, Individual-level cost— 218 yrs with mild to Intervention: Living Life Total QALYs (mean, e Currency: £ Base-case analysis, 
2016,°° utility analysis moderate major to the Full, a coach- by EQ-5D-5L adjusted Year: 2012-2013 with imputations: iCBT 
United : depression (N = 609), guided iCBT, program for baseline utility) with : vs usual care: 
: Mul , , ‘ : : , =e Total : 
Kingdom Becenie ipa primary care setting (n = 307) imputations: ° oe inca) ICER: £132,630 per 
parallel, two-arm, iCBT: 0.541 (SD With imputations: QALY 
individually Mean age: Comparator: usual care “6 009) ( iCBT: 886 INE abawis threshold 
randomized RCT iCBT: 50.0 eau! Usual care: 0.541 (SD Re ee of £20,000/QALY 
NHS and PSS Control: 49.1 Intervention was +0.009) anaes: gained: -£143 (95% Cl 
perspective : provided in 2 stages: : ; ae -164 to -122 
; er Eolas Pre-therapy (max 4 Mean difference iCBT: 864 ) 
Time horizon: 12 iCBT: 69% Usual care: 719 


months 
Discount rate: 0% 


Control: 68% 


>87% were prescribed an 
antidepressant at 
baseline; >91% were 
previously treated for 
depression 


weekly sessions by 
CBT-assistant nurse) 
CBT therapy (max 26 
weekly sessions) 


iCBT: self-directed over 
4 mo, with an option of 
phone support for up to 
10 calls over 12 mo by 
a trained coach (non- 
regulated health care 
professional, 
supervised by nursing 
staff and pharmacists); 
participants also had 
access to a secure web 
portal including health 
information, and access 
to the Big White Wall 
online forum 


adjusted for baseline 
differences, with 
imputations: 0.001 
(95% Cl -0.023 to 
0.026) 


Total QALYs (mean, 
by EQ-5D-5L) 
complete case 
analysis: 

iCBT: 0.535 

Usual care: 0.573 


Mean difference 
adjusted for baseline 
differences, QALY, 
complete case 
analysis: 0.037 (95% 
Cl 0.009-0.066) 


Missing data >50% in 
both effectiveness and 
cost outcomes; data 
imputated using 
multiple imputation 
techniques 


Mean difference: 
With imputations: 168 
(95% Cl 43-294) 
Complete case 
analysis: 145 (95% Cl 
-11 to 300) 


Intervention-related 
costs: 

Training costs and 
encounter (phone) 
calls, non-scheduled 
calls, CBT book (if 
needed), after 
imputation of the 
missing data: £113.0 
Software license: £9.7 
and Phone calls: £71.7 


Probability of iCBT 
being cost-effective 
compared with usual 
care: 

wtp of £20,000 per 
QALY: 30% 

wip of £30,000 per 
QALY: 37%. 


ICER remained robust 
(not cost-effective) in 
scenario analyses that 
exclude the costs of 
the online forum; 
however, ICER was 
cost-effective in the 
complete case 
analysis: £3,850 per 
QALY (>98% 
probability of cost- 
effectiveness at NHS 
wtp threshold) 


The study authors 
questioned the value of 
the complete case 
analysis and consider it 
biased 
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Name, 
Year, Study Design and Interventions / 
Location Perspective Population Comparators Health Outcomes Costs Cost-Effectiveness 
Titov et al, e Individual-level cost— Australian residents 260 Intervention: Managing Total QALYs (mean Currency: AUD Base-case analysis: 
2015, utility analysis yrs with non-severe Your Mood, iCBT plus per person): Year: 2013 ICER: $4,392 per 
Australia -label RCT depression (PHQ-9 email and phone iCBT: 0.114 (95% Cl Total cost QALY (95% Cl <0- 
: ae pba ie scores 10-19), referred support from a senior 0.002-0.15) eer oes e eae 63,962) 
© perspective by GP (N = 54) therapist (n = 27) Control: 0.102 (95% Cl 155.5-292.3) Probability of iCBT 
e — Time horizon: to end Mean Age Comparator: waitlist —0.01 to 0.14) Control: 146.6 (95% Cl being cost-effective 
of treatment (8 wks) iCBT: 64.52 (SD +2.58) control (n = 25) Mean difference: 0.012 80.8-250.6) compared with waitlist 
e _ Discount rate: 0% Waitlist control: 66.16 Breaecionaidalivered (0.004—0.02) Mean difference: 52.04 control at wtp of 


(SD +3.80) 


Females 
iCBT: 81.5% 
Waitlist control: 64% 


Mixed population of 
anxiety disorders, 90% 
major depression and/or 
GAD 

Participants on 
medications: not reported 


over 8 wks plus support 
from psychotherapist 
(10 min weekly) 


(-23.8 to 128.2) 


Total therapist time 
(mean): 45.07 (SD 
+32.51) 


$50,000 per QALY: 
>95% 


Probability of iCBT 
being cost-effective 
compared with waitlist 
control at wtp of $4,392 
per QALY: 50% 
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Name, Methods Results 
Year, Study Design and Interventions / 
Location Perspective Population Comparators Health Outcomes Costs Cost-Effectiveness 
Geraedts et Individual-level cost- Adults, employees, with Intervention: Total QALYs (mean Currency: € Base-case analysis, 
al, 2015, effectiveness, cost— mild to moderate Happy@Work guided - per person) at 12 mo: Year: 2012 societal perspective 
Netherlands utility and cost-benefit depression, CES-D = 25 iCBT iCBT: 0.79 (SD + 0.02) Total costs (mean), ICERs: 

analyses at baseline (N = 231), 6 (n = 116) TAU: 0.78 (SD + 0.02) 532,959/QALY lost 


Open RCT, individual 
block randomization 


Employer and societal 
perspectives 


Time horizon: 
12 mo 


Discount rate: 0% 


companies 


Mean age: 

iCBT: 43 (SD +8.9) 
TAU: 43.8 (SD +9.6) 
Females 

iCBT: 77% 

TAU: 67% 


Comparator: TAU, 
email referral to 
employee (participant) 
with advice to seek 
treatment for their 
symptoms (n = 115) 


iCBT: minimal guidance 
web-based program, 6 
sessions plus one 
booster session, 
homework assigned; 
coach available to 
provide feedback on 
the assignment; a 
tunneled intervention 
allowing participants to 
start with a new 
session only after 
getting feedback on 
their prior homework; 
coaches (MSc students 
trained in clinical 
psychology) used a 
manualized protocol 
and all feedback was 
reviewed by a 
supervisor; all 
participants were 
allowed to seek 
additional mental health 
care 


Mean difference, 
QALY: -0.001 (95% Cl 
-0.04 to 0.04) 


Mean difference: 
Point change in CES- 
D: -2.3 (95% Cl -4.3 
to -0.3) 

Clinically significant 
change: 0.1 (95% Cl 
0.0 to 0.2) 


Total benefits 
(occupational health, 
absenteeism, and 
presentism, mean per 
person), iCBT vs TAU: 
793 


societal perspective: 
CCBT, iCBT: 22,402 
(SD +1,953) 

TAU: 23,115 (SD 
+1,357) 


Mean difference, iCBT 
vs TAU: -714 (-5,018 
to 3,924) 


iCBT cost: 236 per 
user 


(every QALY lost was 
associated with 
savings of 532,959) 
314/point decrease in 
depressive symptoms 
€6,645/ clinically 
significant 
improvement in 
depressive symptoms 


Probability of iCBT 
being cost-effective 
compared with TAU at 
any wtp per QALY: 
62% 


Cost-benefit analysis: 
benefit-to-cost ratio = 
2.8 (P> .05) 

For every € invested, 
the employer received 
€2.8 back for an ROI of 
178% (P> .05); the 
iCBT intervention 
probability of return: 
63% 
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Name, Methods Results 

Year, Study Design and Interventions / 

Location Perspective Population Comparators Health Outcomes Costs Cost-Effectiveness 
Solomon et Cost-utility analysis Adults with mild to Intervention: "My Total QALYs: Currency: AUD Base-case analysis: 
al, 2015, Model-based analysis moderate depression, Compass," unguided iCBT: 0.26 (95% Cl Year: 2013-2014 iCBT vs face-to-face 
Australia anxiety, and/or stress iCBT during acute 0.15-0.34) CBT, iCBT vs TAU: 


NR perspective 


Time horizon: 6 
months 


Discount rate: 0% 


Mean age: not reported 
Females: not reported 


Stepped-care that reflects 
Australian and national 
guidelines within 28 wks: 
7-wk treatment during 
acute phase, and 21-wk 
treatment during 
maintenance phase; 
decision tree reflected 
probabilities of 
compliance to treatment, 
drop-out, relapse, switch 
to other therapy, and 
initial remission and 
remission in the 
maintenance phase 


phase plus 21-wk 
maintenance phase 
consisting of booster 
internet-delivered 
program (monitoring of 
symptoms, behaviours, 
and lifestyle factors) 


Comparators: 
Face-to-face CBT with 
a clinical psychologist 
for acute phase over 7 
wk, and 

TAU: drug treatment in 
the acute phase and 
after remission in the 
maintenance phase 


CBT: 16 sessions 
during the 3-mo acute 
phase and 2 "booster" 
sessions after 


Model: 

Participants not 
compliant to iCBT 
switch to TAU or face- 
to-face CBT 
Participants not 
compliant to face-to- 
face CBT switch to 
TAU (drugs: most 
commonly prescribed 
antidepressants based 
on administrative data) 
Participants not 
responding to TAU 
switch to another drug; 
those who switch 
treatment either enter 
remission or 
discontinue treatment 
All patients have 
consultations with GP 


Face-to-face CBT: 
0.29 (95% Cl 0.16— 
0.37) 
Pharmacotherapy 
(TAU): 0.24 (95% Cl 
0.15—-0.32) 


Mean difference, iCBT 
vs face-to-face CBT 
compared to TAU: 0.02 
and 0.03 


Total costs (mean): 
iCBT: 334.96 (95% Cl 
332.01-338.75) 
Face-to-face CBT: 
2,330.51 (95% Cl 
2,201.10-2,408.40) 
TAU: 3,645 (95% Cl 
457.05-619.77) 


Mean difference, iCBT 
vs face-to-face CBT, 
compared to TAU: 
1,996 and 158 


ICER (face-to-face 
CBT vs iCBT): AUD 
66,518/QALY; ICER 
(iCBT vs 
pharmacotherapy): 
cost-saving 


Probability of iCBT 
being cost-effective at 
wtp of $50,000 per 
QALY: 75.5% when 
compared to both TAU 
and face-to-face CBT 
97%, against TAU 97% 
at any threshold value 


Face-to-face CBT is 
the most efficient at 
wtp of $65,000 per 
QALY: 80.3%. 


Sensitivity analyses: 
EVPI: $79.37 per 
patient 

EVPPI: the rate of non- 
adherence to iCBT 
needs further research 
iCBT would not be 
cost-effective at 
$50,000/QALY if the 
cost of iCBT increased 
from $56 to $309 or if 
the cost of face-to-face 
CBT decreased from 
$734 to $309 
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Phillips et al, 
2014," 
United 
Kingdom 


Gerhards et 
al, 2010,” 
Netherlands 


Individual-level cost— 
utility analysis 


A phase 3 single-blind 
RCT 


Societal perspective 
Time horizon: 1 year 
Discount rate: 0% 


Individual-level cost— 
utility and cost- 
effectiveness 
analyses 


Open RCT 
Societal perspective 


Time horizon: 12 
months 


Discount rate: 0% 


Adults with mild 
depression (N = 637) 


Mean age (overall): 

43 yr 

Females (overall): 48% to 
50% 


18-65 yr with mild to 
moderate depression (N 
= 303), primary care 
setting 


Mean age: 

iCBT: 44.3 (SD +11.8) 
iCBT plus TAU: 45.2 (SD 
+10.9) 

TAU: 45.1 (SD +12.2) 


Females 

iCBT: 52% 

iCBT plus TAU: 63% 
TAU: 55.3% 


at prescribed intervals 
(2-3 visits per cycle) to 
monitor their 
symptoms; some have 
monthly psychiatric 
consults (based on an 
average rate of 0.3% 
and 10.6%) 


Intervention: 
MoodGYM, unguided 
iCBT 

(n = 318) 
Comparator: self-help 
via mental health 
websites (n = 319) 


Five 1-hr sessions of 
CBT, delivered weekly 
via software application 


Interventions: Color 
Your Life, unguided 
iCBT (n = 100) vs 
unguided iCBT plus 
TAU (n = 100) 


Comparator: TAU (n = 
103) 


iCBT: 8 sessions plus 
one booster session, 
no therapist support 


Usual care consisted of 
4—5 biweekly 
consultations with a 
GP, antidepressants 
added if indicated 


Total QALYs (mean 
per person): 
iCBT: 0.170 
No treatment: 0.167 


Mean difference: not 
reported 


Total QALYs (mean 
per person, corrected 
for baseline utility): 


iCBT: 0.71 (SD +0.17) 
iCBT plus TAU: 0.71 


(SD +0.14) 
TAU: 0.72 (SD +0.16) 


Mean difference, 
QALY: 0.01, P= .842 


Currency: £ 
Year: 2010 


Total costs at 12 
weeks: 

iCBT: 143 

Usual care: 119 


Mean difference: not 
reported 


Currency: € 
Year: 2007 


Total costs (mean, 
societal perspective): 
iCBT: 9,457 (7,547— 
11,506) 

iCBT plus TAU: 10,793 
(8,412-13,328) 

TAU: 11,244 (9,206— 
13,419) 


Mean difference, iCBT 
vs TAU: -711 (-3,111 
to 1,780) 

iCBT plus TAU vs 
TAU: 738 (-1,871 to 
3,477) 

iCBT vs iCBT plus 
TAU: -1449 (-4,309 to 
1282) 


iCBT cost: €50 per 
user 


Base-case analysis: 
ICER: NR (calculated): 
8,000/QALY 


Base-case analysis, 
ICER not reported 


Probability of iCBT 
being cost-effective 
compared with TAU: 
65% at wtp of €0 per 
QALY 

40% at wtp of €80,000 
per QALY; at this 
threshold TAU 
intersects and has the 
same 40% probability 
of cost-effectiveness 


Results remained 
similar in sensitivity 
analyses 
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Methods Results 
Name, 
Year, Study Design and Interventions / 
Location Perspective Population Comparators Health Outcomes Costs Cost-Effectiveness 
Warmerdam Individual-level cost— 218 yr with major Intervention: iCBT (n = Total QALYs (mean Currency: € Base-case analysis, 
et al, 2010,* utility and cost- depression (N = 263) 88), internet-based per person) Year: 2007 iCBT vs waitlist control: 
Netherlands effectiveness Mean age: 45 yr (SD: PST (n = 88) iCBT: 0.16 (95% Cl Total costs (mean), ICER: €22,609 per 
analyses +12.1) Comparator: waitlist 0.152—0.169) iCBT: 2,814 (SD QALY; €1,817 per one 
Open-label block- l(n =87 PST: 0.16 (95% Cl +2 683 additional reliably 
pen-label bloc! Females: 71% control (n = 87) ie +2,683) : ‘ae 
randomized RCT arian 0.192 100-166) PST: 2,705 (SD proved Panicipanit 
; iCBT: 8-session course Control: 0.15 (95% Cl +2,851) (clinically significant 
Societal perspective delivered weekly, plus -0.142 to 0.159) a ; ; change in depression 
Time horizon: 12 apes session after Mean difference, iCBT bernie ag symptom severity) 
weeks : : vs waitlist control and eee Probability of iCBT 
PST: 5 sessions re . (2,814 to 2,558) 5 : 
Discount rate: 0% delivered weekly PST vs waitlist control: PST vs waiting list: 147 being cost-effective 
: ; 0.01 (2,705 to 2,558) compared with waitlist 
Both interventions: : ’ control at wip of 
participants supported Total treatment costs: €30,000 per QALY: 
by life coach via email, iCBT: 501 52% 
therapists spent 20 PST: 338 
minutes per week on 
each participant 
Hollinghurst Individual-level cost— 18-75 yr with major Intervention: therapist- Total QALYs (mean Currency: £ Base-case analysis: 
et al, 2010, utility and cost- depression (N = 297), guided iCBT plus usual per person): Year: 2007 iCBT vs waitlist control, 
United effectiveness primary care setting care (n = 149) Therapist-guided iCBT Total costs (mean): ICER: 17,173 per 
Kingdom analyses plus usual care: 0.522 QALY; 3528 per extra 


Double-blinded 
multicentre RCT 


Societal perspective 


Time horizon: 8 
months 


Discount rate: 0% 


Mean age: 34.9 (SD 
+11.6) 


Females: 68% 


>50% were prescribed an 
antidepressant 


Comparator: usual 
care, waitlist control (n 
= 148) 


iCBT: up to 10 
sessions, 55 mins 
each, over 4 mo; 
therapist supported 
with access to on-line 
psychologist; 


Mean number of on-line 
sessions: 6.1 (SD +3.8) 


(SD +0.012) 
Usual care: 0.495 (SD 
+0.016) 


Mean difference, 
QALY: 0.027 (-0.012 
to 0.066) 


% recovered (BDI 
<10): 
Therapist-guided iCBT 
plus usual care: 31% 
Usual care: 19% 


Mean difference, % 
recovered: 13.3 (-3.4 
to 30) 


iCBT: 764 (SD +380) 
Usual care: 295 (SD 
+359) 


Mean difference, iCBT 
vs usual care: 469 
(95% Cl 342-597) 


Total treatment costs: 
iCBT: 493 (SD +185); 
therapists were paid 
40/hr, on-line 
psychologist service 
was paid 11; overhead 
charge was negotiated 
11 (would be £20); 
supervision cost: 
62.50/hr 


participant 


Probability of iCBT 
being cost-effective 
compared with waitlist 
control: 

56% at wtp of 20,000 
per QALY 

71% at wtp of 30,000 
per QALY 


ICER was below the 
base case estimate in 
sensitivity analyses 
(accounting for 
imputed data, 
excluding hospital 
costs, and with max 
costs of iCBT) 
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Name, Methods Results 
Year, Study Design and Interventions / 
Location Perspective Population Comparators Health Outcomes Costs Cost-Effectiveness 
Kaltenthaler Individual-level cost— Adults with major Interventions: e ~—- Total QALYs: e Currency: £ Base-case anaysis 
et al, 2006,°° utility and cost- depression and/or Beating the Blues (for Beating the Blues: 1.10 Year: NR (ICER): 
United effectiveness anxiety, non-severe major depression Cope: 1.05 ¢ — Total costs (mean): Beating the Blues vs. 
Kingdom analyses depression, or and/or anxiety) Overcoming Beating the Blues: 584 TAU: 1801 per QALY 
Model-based analysis depression Cope (for non-severe depression: 1.03 Cope: 630 Cope vs. TAU: 7139 
expanding the results Mean age: NR depression) TAU: 1.02 Overcoming per QALY 
of 3 RCTs Femaise: NA eee * Mean difference: depression: 501 Overcoming 
: (for depression) Beating the Blues vs. TAU: 437 depression vs TAU: 


NHS 


Time horizon: 18 
months 


Discount rate: 3.5% 


Model accounted for 
different severity levels of 
depression and 
associated costs: licence 
fees, computer hardware, 
screening of patients for 
suitability, clinical 
support, capital overhead 
(for facilities for computer 
and clinician) and the 
training of staff 


Usual GP care 
according to NICE 
guidelines, stepped- 
care approach) 


Beating the Blues: a 
15-min introductory 
video followed by eight 
50-minute therapy 
sessions, with 
homework exercises 
between sessions 
Cope: 3-mo program 
with 5 main treatment 
modules 

Overcoming 
Depression: a CD- 
ROM-based CBT 
system, six 45—50 min 
sessionswith support 
from a nurse-clinician 


usual care: 0.08 
Cope vs. usual care: 
0.03 

Overcoming 
depression vs usual 
care: 0.01 


Mean difference, 
Beating the blues vs 
usual care by 
depression severity: 
Mild: 0.07 
Moderate: 0.08 
Severe: 0.08 


Mean difference: iCBT 
vs waiting list: 256 
(2,814—2,558) 

PST vs waiting list: 147 
(2,705—2,558) 


Mean difference vs. 
usual care: 

Beating the Blues: 147 
Cope: 193 
Overcoming 
depression: 64 


Mean difference, 
Beating the blues vs 
usual care by 
depression severity: 
Mild: 131 

Moderate: 157 
Severe: 154 


Estimated costs per 
participant: 

Beating the Blues, one 
copy: 219.3 (152.4— 
353.0) 

Cope, home access: 
171.3 (122.7-268.2) 
Overcoming 
depression, one copy: 
72.6 (42.4-133.0) 


5391 per QALY 


Probability of treat 
being cost effective at 
wtp £30,000 per QALY: 
Beating the Blues: 87% 
Cope: 63% 
Overcoming 
depression: 54% 
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Name, Methods Results 
Year, Study Design and Interventions / 
Location Perspective Population Comparators Health Outcomes Costs Cost-Effectiveness 
McCrone et e Individual-level cost— 18-75 yrs with prevalent Intervention: 1) a Total QALYs (mean), Currency: £ e __ Base-case analysis: 
al, 2004,%? utility and cost- or new major depression commercial pay-to-use iCBT plus usual care Year: 1999-2000 ICER: 40 per unit 
United effectiveness or anxiety who are not iCBT program (Beating vs usual care: NR Total costs (mean) at 8 reduction, 1,333/QALY 
ingdom emo te Meee ae rca The mean difference, mo: * Probability of Beating 
e Open RCT, primary os i depression-free days, Beating the Blues: 397 the Blues being cost- 
care setting involving Mean age: Comparator: Usual GP iCBT vs. TAU: 28.4 (SD +589) effective: 
12 GP practices in Beating the Blues: 43.6 care (n = 138) (10.7-45.5) Usual care: 357 (SD 80% at wtp 0 per unit 
south-east England (SD +14.4) Beatin 2 F +575) reduction 
: g the Blues: a The estimated mean a 
e NHS perspective Control: 43.7 (SD +13.7) 15-minute introductory difference (QALYs), Mean difference: 40 Pe wip 5,000 per 
Females: video followed by eight iCBT vs. TAU: 0.032 (-28 to 148) Q 


e Time horizon: 8 
months 


e Discount rate: 0% 


Beating the Blues: 73% 
Control: 74% 


50-minute therapy 
sessions with 
homework exercises 
between sessions 


Beating the Blues 
software license per 
session was estimated 
at 14.50 (5-30 in 
sensitivity analysis) 


Abbreviations: CCBT, computerized CBT; EVPI, expected value of perfect information; EVPPI,expected value of partial perfect information; GP, general practitioner; iCBT, internet-delivered cognitive behavioural therapy; 

ICER, incremental cost-effectiveness ratio; INB, incremental net benefit; NHS, National Health Service; NICE, National Institute of Health and Care Excellence; NR, not reported; PSS, personal social services; PST, problem-solving 
therapy; QALY, quality-adjusted life-year; RCT, randomized controlled trial; REEACT, Randomised Evaluation of the Effectiveness, cost-effectiveness and Acceptability of Computerised Therapy; ROI, return on investment; SD, standard 
deviation; SD, standard deviation; TAU, treatment as usual; wtp, willingness to pay. 
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Table 5: Results of Economic Literature Review—Cost-Effectiveness of iCBT in the Treatment of Anxiety Disorders 


Name, Methods Results 

Year F F 

bacatlon Be eacnoee Population eta Health Outcomes Costs Cost-Effectiveness 
Kumar et al, Cost-utility analysis Adults with GAD with or Intervention: coach- Total QALYs (per Currency: USD Base-case analysis, 
2018, without comorbidities guided iCBT, "Lantern": 100,000), iCBT vs. Year: 2016 iCBT compared with 


United States 


Decision analytic 
(modeling) analysis 


Health care payer 
perspective and 
societal perspective 


Time horizon: lifetime 
Discount rate: 3% 


(GAD-7 scores: 5-9 
(mild), 10-14 (moderate), 
15-21 (severe) 


Mean age (overall): NR 
Female (overall): NR 


Modeled: Markov model, 
3-mo cycle length, 8 
health states (no anxiety, 
mild, moderate/severe 
anxiety without/with 
comorbidities [4 separate 
states], death); health 
state utilities linearly 
extrapolated from SF-6D 
scores elicited in adults 
with GAD, non-specific to 
CBT; extrapolation of 3- 
mo treatment effects over 
5 yr, assuming a gradual 
decrease of the lifetime; 
risk of suicide 
independent of 
treatments associated 
with major depression, 
based on literature and 
modeled in the states 
combined with 
comorbidities 


1) prevention (all 
patients starting with 
mild anxiety (GAD-7: 
5-9); 2) treatment 
(patients starting with 
moderate or severe 
anxiety, GAD-7: >10- 
21), 3) both prevention 
and treatment 


Comparators: 

1) TAU: 
pharmacotherapy 
(58.6% of the cohort) or 
nothing; 2) face-to-face 
CBT (12-20 1-hr 
sessions) 


iCBT: 3-mo program 
with 8 CBT modules 
accessible over the 
mobile application, 
support provided by a 
trained coach via 
asynchronous 
messages; pilot test 
included 89 employees; 
iCBT program assumed 
equivalent to face-to- 
face CBT 


face-to-face CBT vs. 
TAU: NR 


Mean difference, iCBT 
vs. face-to-face 
CBT/iCBT vs. TAU for 
3 scenarios: 1) base 
case (both): 
34,108/81,492 QALYs; 
2) prevention only: 
28,959/76,568 QALYs; 
3) treatment only: 
36,564/83,841 QALYs 


Total costs (mean per 
100,000, in billion $), 
iCBT vs. face-to-face 
CBT vs. TAU: 1) payer 
perspective, 
both/prevention/treatm 
ent: 24.9 vs. 25.3 vs. 
25.5/24.9 vs. 25.2 vs. 
25.4/24.9 vs. 25.3 vs. 
25.6; 2) societal 
perspective, 
both/prevention/treatm 
ent: 36.5 vs. 38.7 vs. 
41.0/36.3 vs. 38.2 vs. 
40.4/36.6 vs. 38.9 vs. 
41.3 


Mean difference (per 
100,000, in billion $), 
iCBT vs. face-to-face 
CBT, and iCBT vs. 
TAU: 1) payer 
perspective, 
both/prevention/treatm 
ent: a) 339/297/360; b) 
605/553/630; 2) 
societal perspective, 
both/prevention/treatm 
ent: a) 339/297/360; b) 
605/553/630 


face-to-face CBT or 
TAU (pharmacotherapy 
alone), payer or 
societal perspective: 
cost saving (higher 
benefits, lower costs) 


Sensitivity analysis: 
PSA not conducted; a 
single deterministic 
sensitivity analysis 
showed that face-to- 
face CBT became as 
cost-effective as iCBT 
when clinical response 
rate in face-to-face 
CBT increased to 76% 
(from 42% in base 
case analysis), while 
cost-effectiveness 
remained the same for 
iCBT (38%) 
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R I 
Name, Methods esults 
Year, Study Design and Interventions/ 
Location Perspective Population Comparators Health Outcomes Costs Cost-Effectiveness 
EI Alaoui et e Individual-level cost- e ~—- Adults with SAD (N = e Intervention: guided Total QALYs (mean Currency: € Base-case analysis, 
al, 2017,°' minimisation analysis, 126) iCBT modules including per person), iCBT vs. Year: 2017 the cost-minimization 
Sweden continues previous M -NR homework exercises (n group face-to-face Totalcoststancd analysis, at 80%, 50%, 
RCT by Hedman et al Bae = 64) CBT: NA (equivalence Ere fea or 100% capacity: 
(2014) ° Female: NR ° Comparator: ecm in iCBT vs. group CBT, aan cost saving in all 
e —Noninferiority RCT, 4- face-to-face group CBT prior RCT) 80% capacity (base hE AES 
yr follow-up (n = 62) Mean difference: NA case): 463 (446-480) 
e Health care payer e _ iCBT: online modules, vs. 806 (730-883) 
perspective 15 wk, with access toa Mean difference, at 
‘ ae therapist and 80% capacity: -343 
Time h : 15 wk ; ; 
i ieenonues us supportive email (-267 to -420) 
; feedback (time spent Rasclroeise 
e Discount rate: 0%, with a patient: 10 i i 
3% 5% in/ n estimated using a 
: min/wk) bottom-up approach, 
e Group iCBT: 15 costing clinical and 
sessions, 1 individual administrative activities 
and 14 weekly group during the treatment 
CBT sessions, 2.5 hr delivery cycle 
long, groups of 5-7 
patients, led by CBT- 
trained therapists 
Hedman et e Individual-level cost- e Adults with severe lasting ¢ Intervention: iCBT Total QALYs (mean Currency: USD ICER of base-case 
al, 2016, effectiveness and health anxiety modules including per person), iCBT vs. Year: 2013 analysis: $10,000 per 
Sweden cost-utility analysis (hypochondriasis), homework exercises (n control, post-treatment: QALY 


RCT 

Societal perspective 
Time horizon: 12 wk 
Discount rate: 0% 


primary care setting (N = 


158) 


Mean age, iCBT vs. 
control: 41.7 (SD: 13.6) 
vs. 41.4 (SD: 13.2) yr 


Females, iCBT vs. 
control: 81% vs. 77% 


Anxiety present for 
around 13 yr 


= 79) 


Comparator: 

Control internet 
delivered behavioural 
stress management 
program (n = 79) 


iCBT: 12 modules over 
12 wk, with access to a 
therapist via a secure 
online contact system 


NR 
Mean difference: 0.031 


Total costs (mean), 
iCBT vs. control, post- 
treatment: 1,607 (SD: 
1,698) vs. 1,340 
(SD:1,123) 


Mean difference: 310 


Cost of treatment: 571 
vs. 431 


Probability of iCBT 
being cost-effective 
compared with control 
at a wtp per QALY: NR 
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Name, Methods Results 

Year, Study Design and Interventions/ 

Location Perspective Population Comparators Health Outcomes Costs Cost-Effectiveness 
Dear et al, e Individual-level cost- Adults aged 60 yr or older Intervention: supported Total QALYs (mean Currency: USD ° ICER for base-case 
2015, effectiveness analysis with anxiety (GAD-7 28), iCBT program per person), iCBT vs. Year: 2013 analysis: $8,806 per 
Australia residents of Australia (N "Managing Stress and control: 0.102 (0.044— QALY 


Open RCT 


Australia, health 
sector perspective 


Time horizon: 2 mo 
Discount rate: 0% 


= 72) 


Mean age, intervention 
vs. control: 65.4 (SD: 4.7) 
vs. 65.5 (SD: 5.8) yr 


Females, intervention vs. 
control: 67% vs. 54% 


With previous mental 
health treatment, 
intervention vs. control: 
54% vs. 49%; taking 
medications: 24% vs. 
38% 


Anxiety Course" (n = 
75) 


Comparator: 
TAU: Usual GP care (n 
= 64) 


iCBT supported by a 
Clinical psychologist via 
email or phone on 
weekly basis; iCBT 
consisted of 5 lessons 
delivered over 8 wk 


Outcomes: QALYs and 
costs; iCBT costs 
included therapist's 
time, supervisors's time 
(1-hr supervision 
weekly), internet 
access, computer and 
telephone use but not 
cost of software; other 
health care resource 
use: medications, GP 
and psychiatric 
consultations, and 
admissions 


0.139) vs. 0.092 
(0.033-0.128) 


Mean difference: 0.01 
(0.003-0.018) 


Total costs (mean), 
iCBT vs. control: 229.5 
(184.9-276.4) vs. 
137.4 (98.4-173.5) 


Mean difference: 92.2 
(38.7-149.2) 


Costs of guided iCBT 
associated per patient: 
£42 


Probability of iCBT 
being cost-effective 
compared with control 
at wtp of $50,000 per 
QALY: >95% 
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Name, 
Year, Study Design and Interventions/ 
Location Perspective Population Comparators Health Outcomes Costs Cost-Effectiveness 
Nordgren et e __Individual-level cost- Adults with an anxiety Intervention: guided Total QALYs (mean Currency: USD Base-case analysis, 
al, 2014,5° effectiveness and disorder by DSM-IV with iCBT, individually per person by EQ-5D), Year: NR iCBT vs. TAU, ICER: 
Sweden cost-utility analysis or without comorbid tailored (n = 50) iCBT vs. control: NR -$7,523 per QALY; 


e Open RCT 
e Sweden, societal 
perspective 


e Time horizon: 10 wk 
e Discount rate: 0% 


problems, primary care 
setting (N = 100) 


Mean age, iCBT vs. 
control: 35 (SD:13) vs. 36 
(SD:12) yr 


Females, iCBT vs. 
control: 33% vs. 30% 


Prior psychotherapy: 
66%-68%, ongoing 
medication: 24%—28%, 
most commonly 
diagnosed with GAD or 
panic disorder (64%), 
56%—-60% with any 
comorbidity 


Comparator: 

TAU for 10 wk, allowed 
cross-over after 10 wk 
(n =50) 


iCBT: 10 modules with 
homework, delivered 
weekly over 10 wk, 
supported by a 
therapist via internet 
messenger system, 
support provided within 
24 hr and feedback on 
homework ($15 min/wk 
per client), cost per 
client: $507 


TAU: weekly email from 
a therapist related to 
well-being, no specific 
feedback (unless the 
therapist judged the 
further action was 
required), cost per 
client: $68 


Mean difference: 0.063 


Total costs (mean), 
iCBT vs. control, pre- 
treatment/post- 
treatment: $2,648 (SD: 
1,812)/$1,757 (SD: 
1,870) vs. $1,803 
(SD:1,694)/$2,078 
(SD:1,868) 


Mean difference: 
-$474 


Significant time-group 
interaction: iCBT group 
had larger indirect cost 
reduction vs. control 


cost saving and more 
effective 


Probability of iCBT 
being cost-effective 
compared with control 
at wtp of $0 per QALY 
was 90%, rising above 
95% at $3,000/QALY 


Sensitivity analysis: if 
iCBT increased by 
$600, probability of 
iCBT being cost- 
effective compared 
with control at wtp of 
$0 per QALY was 37% 
and was 60% at 
$3,500/QALY 
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Name, Methods Results 

Year, Study Design and Interventions/ 

Location Perspective Population Comparators Health Outcomes Costs Cost-Effectiveness 
Hedman et Individual-level cost- Adults with SAD (N = Intervention: guided Total QALYs (mean), Currency: USD Base-case analysis, 
al, 2014, effectiveness and 126) iCBT modules including iCBT vs. group CBT, Year: NR iCBT vs. group CBT, 
Sweden cost-utility analysis homework exercises (n within group before ICER: -$7,345 per 


Noninferiority parallel- 
group RCT, 4-yr 
follow-up 


Societal perspective 


Time horizon: 4 yr 
(mean: 4.2, range: 
3.1-5.3 yr) 
Discount rate: NR 
(0%) 


Mean age, iCBT vs. 
group CBT: 35.2 
(SD:11.2) vs. 35.5 (SD: 
11.6) yr 


Females, iCBT vs. 
control: 37.5% vs. 33.9% 


Anxiety present for 
around 20 yr, 16% with 
comorbid major 
depression; 34% of 
patients in iCBT and 38% 
of patients in group CBT 
received additional 
treatment (CBT, 
psychological treatment, 
or medication) after 
finalizing the intervention 
in year 1 


= 64) 


Comparator: face-to- 
face group CBT (n = 
62) 


iCBT: online modules, 
15 wk, with access toa 
therapist (psychologist) 
and supportive email 
feedback via text-based 
messaging system with 
no phone/face-to-face 
contact; on average 9 
of 15 modules 
completed, cost per 
treated patient: $464 


Group iCBT: 15 
sessions, 1 individual 
and 14 weekly group 
CBT sessions including 
homework between 
sessions, 2.5 hr long, 
groups of 6-7 patients, 
led by CBT-trained 
experienced 
psychotherapists; on 
average 9 of 15 
sessions completed, 
cost per treated patient: 
$2,687 


intervention to 4-yr 
follow-up: -0.36 (-0.70 
to -0.01) vs. -0.25 
(-0.60 to 0.10) 


Mean difference, iCBT 
vs. group CBT: 0.11 


Total QALYs (mean), 
iCBT vs. group CBT, 
between group before 
4-yr follow-up: -0.18 
(-0.53 to 0.17) 

No significant 


interaction of group 
and time 


Total costs (mean), 
iCBT vs. group CBT, 
before and at 4-yr 
follow-up: before 


treatment, 14,096 (SD: 


14,952) vs. 15,546 

(SD: 15,852); after 4 
yr, 6,349 (SD: 8,366) 
vs. 8,532 (SD: 7,464) 


Mean difference: -808 


QALY; cost saving and 
more effective 


Probability of iCBT 
being cost-effective 
compared with group 
CBT at wtp of $0 per 
QALY is 62% and at 
wtp of $100,000 per 
QALY is 64% 
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Name, Methods Results 
Year, Study Design and Interventions/ 
Location Perspective Population Comparators Health Outcomes Costs Cost-Effectiveness 
Hedman et Individual-level cost- e —_ Adults with severe lasting Intervention: iCBT Total QALYs (mean e Currency: £ Base-case analysis: 
al, 2013,°” effectiveness and health anxiety and no modules including per person), iCBT vs. Year: 2010 ICER: £-6533 per 
Sweden cost-utility analysis history of homework exercises (n control: NR ¢ Total costs (mean) QALY; cost saving 
RCT lea aad = 40) Mean difference: 0.12 iCBT vs. control: NR Probability of iCBT 
F ; = Comparator: M diff --784 being cost-effective 
uel caus e Mean age, iCBT vs. Online discussion : Seneca compared with control 
Time horizon: 1 yr control: 39.3 (SD:9.8) vs. forum (n = 41) at wtp of £5,000 per 
Discount rate: 0% 38.8 (SD: 9.5) yr iCBT: 12 modules of QALY: 77% 
° Females, iCBT vs. sessions taken over 12 
control: 70% vs. 78% wk, with access to a 
* Anxiety present for therapist via a secure 
online contact system 
around 20 yr : : 
(time spent with a 
patient: 9 min/wk): cost 
of treatment £210 
National Cost-utility analysis e —_ Adults with social anxiety Interventions: guided Total QALYs (mean e = Currency: £ Base-case analysis: 
Collaborating Model-based analysis disorder (1.5% of all and unguided iCBT per 1000 persons), Year: 2012 ICER (calculated): 1) 
Centre for Ao iees anxiety disorders) Comparators: waitlist iCBT vs. control: 1) ° Total costs (mean), guided iCBT vs. 
Mental Health an *  Meanage: NR guided: 3,540; 2) F : waiting list: 300 per 
‘ ge: control, drug d : iCBT vs. control: 1) : ‘ 
and NICE, perspective (sertraline), placebo unguided iCBT: 3,848; guided: 4.87 million; 2) QALY; 2) unguided 
2013,% Time horizon: 12 wk * Females: NR i 3) drug: 3,490; 4) unguided iCBT: 4.74 iCBT vs. waiting list: 
United of intervention plus5 «Hybrid decision tree: waiting list: 3,366; 5) million; 3) drug: 4.27 1071 per QALY 
Kingdom yr of follow-up Markov model compared placebo: 3,401 


Discount rate: NR 


28 interventions including 
drugs and iCBT; utilities: 
0.866 associated with 
recovery and 0.659 with 
non-recovery, relapse 


Mean difference: NR 


million; 4) waiting list: 
4.29 million; 5) 
placebo: 4.396 million 


Mean difference: NR 


Total costs of 
iCBT/drug include 
costs of visits to GP: 1 
for iCBT and 7 for 
drugs: 1) guided: 877; 
2) unguided iCBT: 649; 
3) drug: 324; waiting 
list: O 
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Name, Methods Results 
Year, Study Design and Interventions/ 
Location Perspective Population Comparators Health Outcomes Costs Cost-Effectiveness 
Joesch et al, e Individual-level cost- Adults aged 18-75 yr, Intervention: guided Total QALYs (mean), Currency: USD Base-case analysis 
2012,°° effectiveness and English- or Spanish- iCBT modules (CALM) iCBT vs. TAU, baseline Year: 2009 (complete case 


United States 


cost-utility analysis 


Parallel-group RCT, 
17 clinics in 4 US 
cities (Arkansas, 
California, and 
Washington) 


Health care payer 
perspective 


Time horizon: 18 mo 
Discount rate: 0% 


speaking, with an anxiety 
disorder (moderate 
anxiety) by DSM-IV (N = 
690) 

Mean age, iCBT vs. TAU: 
44.7 (SD:12.8) vs. 45.6 
(SD: 13.6) yr 


Females, iCBT vs. 
control: 72% vs. 71% 


75%—-78% had GAD, 
62% with comorbid major 
depression at baseline; 
57%-58% had more than 
2 comorbid conditions 6 
mo before baseline: 4 
primary care visits and 1 
ER visit, > 1 specialist 
visit 


provided within a 
collaborative-care 
case-management 
model offered a choice 
of CBT, anti-anxiety 
medication, or both (n = 
349) 


Comparator: 

TAU (n = 341) 

iCBT within 
collaborative care 
model: online CALM 
modules, 10-12 wk, 
supported by non- 
expert care manager; 
care managers 
involved in care: 
assisted with treatment 
adherence, medication 
optimization, and 
consults with 
psychiatrists as 
needed; duration of 
treatment 3-12 mo, first 
course of 3 mo; if 
patient wanted, could 
repeat course up to 3 
times at 3-mo intervals; 
after finishing 
treatment, 1-mo calls 
from managers 
reinforced CBT skills, 
drug adherence, or 
both 


TAU: Usual care by 
GP, medication, 
counselling, or referral 
to a mental health 
specialist 


to 18 mo: EQ-5D, 1.17 
(95% Cl 1.14—1.19) vs. 
1.11 (1.09-1.14); SF- 
6D, 1.05 (95% Cl 
1.04-1.07) vs. 1.00 
(0.98—-1.02) 


Mean difference, iCBT 
vs. TAU: EQ-5D, 0.05 
(95% Cl 0.01—0.09), 
SF-6D, 0.05 (95% Cl 
0.03-0.08) 


Total costs (mean), 
iCBT vs. TAU, baseline 
to 18 mo: 7,310.5 (95% 
Cl 6,669-7,951) vs. 
7,065.7 (95% Cl 
6,325—7,806) 


Mean difference: 244.8 
(95% Cl -733 to 1,223) 


analysis, non- 
parametric, for EQ-5D), 
iCBT vs. TAU, INB > 0 
at wtp of 5,000, 95% Cl 
around INB > 0 at 
60,000 


Scenario analysis 
(missing data 
imputation analysis, 
non-parametric, for 
EQ-5D), iCBT vs. TAU, 
INB > 0 at wtp of 
10,000, 95% Cl around 
INB > 0 at 80,000 
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Name, Methods Results 
Year, Study Design and Interventions/ 
Location Perspective Population Comparators Health Outcomes Costs Cost-Effectiveness 
Hedman et Individual-level cost— Adults with SAD (N = Intervention: guided Total QALYs (mean), Currency: USD Base-case analysis, 
al, 2011,°8 utility and cost- 126) iCBT modules including iCBT vs. group CBT, Year: 2009 iCBT vs. group CBT, 
Sweden a a Mean age, iCBT vs. peers exercises (n fee vs. group CBT, F Total costs (mean), ee eee per ; 
analyses group CBT: 35.2 (SD: = 64) pre See Set an iCBT, iCBT plus TAU, Q 5 cost saving an 
Noninferiority parallel- 11.2) vs. 35.5 (SD: 11.6) Comparator: 6 mo: before & TAU (societal more effective 
group RCT yr face-to-face group CBT treatment, 0.77 (S perspective): 9,457 Probability of iCBT 


Societal perspective 
Time horizon: 6 mo 
Discount rate: 0% 


Females, iCBT vs. 
control: 37.5% vs. 33.9% 


Anxiety present for 
around 20 yr, 16% with 
comorbid major 
depression; 34% of 
patients in iCBT and 38% 
of patients in group CBT 
received additional 
treatment (CBT, 
psychological treatment, 
or drugs) after finalizing 
the intervention in year 1 


(n = 62) 


iCBT: online modules, 
15 wk, with access to a 
therapist (psychologist) 
and supportive email 
feedback via internet- 
based messaging 
system with no 
phone/face-to-face 
contact (10 min/wk); on 
average 9 of 15 
modules completed, 
cost per treated patient 
was $464, and 5.5 
min/wk of therapist’s 
time 

Group iCBT: 15 
sessions, 1 individual 
and 14 weekly group 
CBT sessions including 
homework between 
sessions, 2.5 hr long, 
groups of 6-7 patients, 
led by CBT-trained 
experienced 
psychotherapists; on 
average 9 of 15 
sessions completed, 
cost per treated patient 
was $2,687 and 50 
min/wk of therapist’s 
time 


0.18) vs. 0.74 (SD 
0.19); 4 mo after 
treatment, 0.82 (SD: 
0.14) vs. 0.80 (SD: 
0.17); at 6 mo after 
treatment, 0.85 (SD: 
0.14) vs. 0.81 (SD: 
0.17) 


Mean difference, iCBT 
vs. group CBT: 0.075 


(7,547-1 1,506) vs. 
10,793 (8,412-13,328) 
vs. 11,244 (9,206-— 
13,419) Total costs 
(mean), iCBT vs. group 
CBT, pre/post- 
treatment and 6 mo: 
before treatment, 7,048 
(SD: 7,476) vs. 7,773 
(SD: 7,926); 4 mo after 
treatment, $6,598 (SD: 
7,337) vs. $8,648 (SD: 
7,539); at 6 mo, $5,616 
(SD: 7,456) vs. $7,650 
(SD: 6,591) 


Mean difference: 
-$1,335 


being cost-effective 
compared with group 
CBT at wtp of 0 per 
QALY is 81% and at 
wtp of >40,000 per 
QALY is 79% 


Scenario analysis: 
costs of treatment only, 
iCBT vs. group CBT, 
ICER: -29,693 per 
QALY; cost saving and 
more effective 
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Narie. Methods Results 
Year, Study Design and Interventions/ 
Location Perspective Population Comparators Health Outcomes Costs Cost-Effectiveness 
National Cost-utility analysis Adults with panic disorder Interventions: 1) Total QALYs (mean Currency: £ Base-case analysis: 
Collaborating Model-based analysis (N = 21,000, 5% using Therapist-guided iCBT per person), iCBT vs. Year: 2009 ICER: 1) model 1: 
Centre for iCBT of 425,000 patients program "Panic control: 1) model 1: Total costs (mean) 7,599/QALY; 2) model 
Mental Health NHS and PSS in the UK) Online"; 2) Guided 85.46 vs. 80.79; 2) iCBT vs. control: 1) 2: 126,849 (saving 
and NICE, perspective Mean age: NR iCBT "Internet model 2: 83.059 vs. del 1: : ~130,000 for one 

9 ; ; ge: i a model 1: 59,429 vs. 

201 1; Time horizon: 12 mo Psykiatri 83.29;3) model 3: 23.933: 2) model 2: QALY lost);3) model 3: 
United i eae Females: NR Pomparators? 85.46 vs. 80.79; 4) 91.756 vs. 61,456:3) 2,216/QALY;4) model 
Kingdom COUR Tate 1° 4 decision tree models: : model 4: 85.46 vs. ; 25 IN9 ve 4: dominant 


active panic-free status or 
not after treatment, 
remain panic-free or 
return to panic state at 
follow-up (52 wk); No 
utility data for specific 
health states panic 
disorder; utilities 
extrapolated from an 
ESEMeD study that 
measured utilities using 
EQ-5D in people with 
panic disorder for more 
than 12 mo and those 
without panic disorder: 
0.76 (95% Cl 0.70-0.82) 
and 0.91 (95% Cl 0.90- 
0.91), respectively 


1) TAU or 2) clinician- 
led CBT 

iCBT: 12-14 wk, 
guided by a therapist 
(email messaging), 
follow-up at 52 weeks 


Model 1: Panic Online 
vs. TAU; Model 2: 
Panic Online vs. 
clinician-led CBT; 
Model 3: Internet 
Psychiatri vs. waiting 
list; Model 4: Internet 
Psychiatri vs. clinician- 
led CBT 


80.79 


Mean difference: 1) 
model 1: 4.67; 2) 
model 2: -0.239 ;3) 
model 3: 5.217;4) 
model 4: 1.25 


model 3: 32,702 vs. 
21,140;4) model 4: 
26,217 vs. 69,567 


Mean difference: 1) 
model 1: 35,496; 2) 
model 2: 30,300; 3) 
model 3: 11,562;4) 
model 4: 43,350 


Intervention costs 
included therapist's 
costs, hardware (4 per 
person per computer), 
capital overheads (27 
per person), license 
fee (none), 
server/website hosting 
costs (negligible) 


Therapist’s costs per 
patient based on time 
spent: 188-443, 
depending on model 
assumptions 


Probability of iCBT 
being cost-effective 
compared with control 
at wtp of 20,000 per 
QALY: 1) model 1: 
92%; 2) model 2: 71%; 
3) model 3: 85%; 4) 
model 4: 95% 
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Economic Evidence 


Name, 
Year, 
Location 


February 2019 


Bergstrom et 
al, 2010,°* 
Sweden 


Single-blinded 
parallel-group RCT 


Societal perspective 
Time horizon: 6 mo 
Discount rate: 0% 


Mean age: NR 
Females: NR 


More than 50% were 
prescribed an 
antidepressant 


homework exercises (n 
= 53) 


Comparator: face-to- 
face group CBT (n= 
60) 


iCBT: 10 modules, 10 
wk, with access to a 
therapist (psychologist) 
and supportive email 
feedback with no 
phone/face-to-face 
contact; on average 6.7 
of 10 modules 
completed, and 35 min 
of therapist’s time per 
patient (mean: 35.4 
min, SD:19.6) 


Group iCBT: 10 
sessions, group CBT 
including homework 
between sessions, 2 hr 
long, led by 2 trained 
psychotherapists; on 
average 8 of 10 
sessions completed, on 
average 6 hr of 
therapist’s time per 
patient 


post-treatment: 60% 
vs. 63% 


Mean difference, iCBT 
vs. group CBT: NR 


Total costs (mean), 
iCBT vs. group CBT, 
post-treatment: 86 vs. 
325 

Mean difference: 239 


Therapist costs per 


patient/group, iCBT vs. 


group CBT, post- 
treatment: 21 vs. 260 


Methods Results 
Study Design and Interventions/ 
Perspective Population Comparators Health Outcomes Costs Cost-Effectiveness 
Individual-level cost- Adults with panic disorder Intervention: guided Total response rate, Currency: € Base-case analysis, 
effectiveness analysis (N = 113) iCBT modules including iCBT vs. group CBT Year: NR iCBT vs. group CBT, 


ICER (calculated): 
7,970 per responder; 
cost saving and less 
effective 


Probability of iCBT 
being cost-saving 
(south-west quadrant) 
compared with group 
CBT: 62% 
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Name Methods Results 
3 
Year, Study Design and Interventions/ 
Location Perspective Population Comparators Health Outcomes Costs Cost-Effectiveness 
Titov et al, Individual-level cost- Adults aged 18 yr or older Intervention: supported e Total YLDs averted Currency: AUD Base-case analysis: 
2009,°° effectiveness analysis with social phobia, iCBT program (calculated using the Year: 2008 ICER: 1,495 per YLD 
Australia participants in Shyness 1 "Shyness program" (n = disability weight averted 
Two RCT: Total costs (mean), 
wonhieTs and Shyness 2 RCTs (N 93) transfer factor for oe pees ie Sensitivity analysis 
Health sector = 193) TAL: social phobia of Li sie pear ae 
erspective * Comparator: TAU: pat Mean difference: 300 pollinating Costs es 
pereP Mean age: NR waitlist control (n = 100) renee oe . é ' 5 group CBT (800 per 
Time horizon: 6 mo . e  iCBT: 6 online sessions osts of treatment: participant) vs. waiting 
Discount rate: 0% rogers over 8-10 wk, Te ENGI = G:2007 hr of clinician time per list, ICER: 5,686 per 
supported by aclinical ¢ Meandifference: patient (3 x 100/hr) YLD averted 
psychologist via email 0.2007 
on fooler basis, , Acceptability of iCBT 
participation in online explored at 6 mo: 44% 
ae Lestat ss who previously used 
eevee pa eee face-to-face CBT for 
tate ae social phobia preferred 
using iCBT (reasons: 
anonymity) 
McCrone et Individual-level cost- Adults with panic disorder  — Intervention: iCBT e Total effects, iCBT vs. Currency: £ Base-case analysis: 
al, 2009, effectiveness and (N = 90) modules "FearFighter" face-to-face CBT vs. Year: NR eee anes 
ral cost-utility analysis Mean age: 38 (SD: 13) yr including homework (n relaxation, the eee Total costs (mean), Te 
ingdom = NR) improvement on the iCBT vs. face-to-face 


Pilot parallel group 
RCT 


Undefined 
perspective 


Time horizon: 6 wk 
Discount rate: 0% 


Females, iCBT vs. 
control: 69% 


Comparator: 1) face-to- 
face CBT (n = NR), and 
2) computer-aided 
relaxation (n = NR) 


iCBT: 6 computer 
modules, self-guided, 
including homework; 
brief clinician in-person 
support at start and end 
of sessions (76 min 
over 10 wk) 


Face-to-face CBT: 6 
sessions, individual in- 
person, 1 hr, including 
homework; relaxation: 
6 computer-guided 
sessions; brief clinician 
in-person support at 
start and end of 
sessions (76 min over 
10 wk) 


main problem 
ratings/global phobia 
rating: 3.95 vs. 3.93 vs. 
0.71, P> .05/2.95 vs. 
3.59 vs. 1.07, P> .05 


Mean difference: NR 


CBT vs. relaxation, 
assuming 196/patient: 
281 (SD: 88) vs. 363 
(SD: 158) vs. 110 (SD: 
33) 


Mean difference, iCBT 
vs. relaxation/iCBT vs. 
face-to-face: 90% Cl, 
143-£198/90% Cl, 
-129 to -31 


Costs not collected, but 
estimated on basis of 
UK guidelines: cost of 
FearFighter, 
196/patient in one GP 
practice, 111/patient in 
primary care trust; CBT 
therapist costs: 69/hr; 
relaxation treatment 
cost, 0/patient 


per point 
improvement/100-175 
per point improvement 
Probability of 
iCBT/face-to-face CBT 
being cost-effective 
compared with 
relaxation was 50% at 
a wtp of 100 per point 
improvement; for all 
values of wtp, iCBT 
had less chance to be 
cost-effective than 
face-to-face CBT 
Cost-effectiveness of 
iCBT would be better if 
the salary of people 
supporting therapy was 
smaller (less well- 
trained clinicians) 
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Name, Methods Results 
Year, Study Design and Interventions/ 
Location Perspective Population Comparators Health Outcomes Cosis Cost-Effectiveness 
Mihalopoulos Cost-effectiveness Adults with panic disorder Interventions: 1) Total DALYs (mean), Currency: AUD e  Base-case analysis: 
et al, 2005,°" analysis (NR, based on 2004 therapist-guided iCBT iCBT vs. usual care: Year: NR ee ' 
peenalia Model-based analysis prevalenes of patie eater ae ne Total costs (mean), eet ae 
: . disorder in Australia) Online"; 2) GP-assisted ‘ . : : 
(using pilot RCT data) iCBT "Panic Online" Mean difference: 870 iCBT by 4,300/DALY 
Lissiih seeing Mean age, iCBT vs. celal DALYs (90% Cl 540- psychologist/iCBT by averted/3,200/ 


perspective 
Time horizon: 6 mo 
Discount rate: 0% 


control: NR 


Females, iCBT vs. 
control: NR 


Comparator: face-to- 
face CBT by a GP or 
usual care (a mixture of 
no care/evidence- 
based principles and 
no-evidence-based 
medicine principles) 


iCBT: 1) 12-wk 
intervention comprising 
iCBT plus 12 weekly 
45-min sessions with a 
publicly funded 
psychologist, and one 
consultation with GP; 2) 
12-wk iCBT plus 6 
consultations with a GP 


1,200) 


Assumed that 
effectiveness of iCBT 
is equal to 
effectiveness of face- 
to-face CBT 


GP vs. usual care: NR 


Mean difference, iCBT 
by psychologist/iCBT 
by GP vs. usual care: 
$3.8 million (90% Cl, 
2.3-5.3 million)/2.8 
million (90% Cl, $1.7— 
3.9 million) 


DALY averted 
Probability of iCBT (by 
a psychologist or a GP) 
being cost-effective 
was 100% at wip of 
10,000 
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Name, 
Year, 
Location 


Kaltenthaler, 
2006,° 


Economic Evidence February 2019 
Methods Results 
Study Design and Interventions/ 
Perspective Population Comparators Health Outcomes Costs Cost-Effectiveness 
Cost-utility analysis Adult patients with panic Interventions, Total QALYs (mean), Currency: £ Base-case anaysis: 
disorder commercial pay-to-use FearFighter vs. Year: NR FearFighter vs. 


United 
Kingdom 


Model-based analysis 
NHS perspective 
Time horizon: 12 mo 
Discount rate: 0% 


Mean age, iCBT vs. 
control: NR 


Females, iCBT vs. 
control: NR 


Two-state Markov model 
with 3-mo cycle length 
(panic phobia or well); 
relapse rate (annual rate 
of 17%, 3-monthly rate of 
0.045) assumed to be 
same for iCBT and face- 
to-face CBT; utilities 
assumed from ESEMeD 
study in patients with 
prior phobia: social 
phobia (0.79, 95% Cl 
0.75—0.84), agoraphobia 
(0.79, 95% Cl 0.73-0.84) 
and specific phobia (0.82, 
95%Cl 0.80—0.85); well, 
no disorder (0.91, 95% Cl 
0.90-0.98) 


programs (ST 
Solutions): FearFighter 


Comparator: 1) iCBT 
for relaxation with brief 
coaching sessions of 5 
min; 2) face-to-face 
CBT (6 hr of individual 
therapy) 


iCBT: FearFighter (6- 
session module) with 
telephone support line 


relaxation vs. face-to- 
face CBT: 0.794 vs. 
0.736 vs. 0.805 


Mean difference, 
FearFighter vs. 
relaxation/face-to-face 
CBT vs. FearFighter: 
0.058/0.011 


Total costs (mean), 
FearFighter vs. 
relaxation vs. face-to- 
face CBT: 217 vs. 78 
vs. 410 


Mean difference, 
FearFighter vs. 
relaxation/face-to-face 
CBT vs. FearFighter: 
138/194 


relaxation/face-to-face 
CBT vs. FearFighter, 
ICER: 2,380 per 
QALY/17,608 per 
QALY 

Probability of 
FearFighter vs. 
relaxation/face-to-face 
CBT vs. FearFighter at 
wtp 30,000 per QALY: 
39%/61% 

Reduction of the cost 
of FearFighter would 
lead to increase in the 
ICER (face-to-face 
CBT vs. FearFighter) 
and better acceptability 
of iCBT program as 
compared with face-to- 
face CBT 


Abbreviations: CALM, Coordinated Anxiety Learning and Management; CBT, cognitive behavioural therapy; CCBT, computerized CBT; DALY, disability-adjusted life-year; DSM-IV, Diagnostic and Statistical Manual of Mental Disorders, 4th 
edition; EQ-5D, European Quality of Life questionnaire in five dimensions; ESEMeD, European Study of the Epidemiology of Mental Disorders; GAD, generalized anxiety disorder ; GP, general practitioner; iCBT, internet-delivered cognitive 
behavioural therapy; ICER, incremental cost-effectiveness ratio; INB, incremental net benefit; NA, not applicable; NHS, National Health Service; NICE, National Institute of Health and Care Excellence; NR, not reported; PSS, personal social 
services PST, problem-solving therapy; QALY, quality-adjusted life-year; RCT, randomized controlled trial; ROI, return on investment; SAD, social anxiety disorder; SD, standard deviation; SF-6D, Short-Form Health Survey in 6 Dimensions; 
TAU, treatment as usual; wtp, willingness to pay; YLD, years lived with disability. 
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Applicability and Limitations of the Included Studies 


Appendix 4 presents the results of the methodology checklist for economic evaluations applied 
to the included articles. (Tables A2 and A3 present data for the 14 evaluated cost-effectiveness 
studies for major depression, and Tables A4 and A5 present data for the 16 evaluated cost- 
effectiveness studies for anxiety disorders.) All studies were deemed partially applicable to our 
two research questions. We considered none of the included studies relevant or directly 
applicable for the Ontario or Canadian setting. 


We also assessed the methodological quality of these studies. Regarding the 14 cost- 
effectiveness analyses of iCBT for major depression, six studies had minor limitations and the 
rest had potentially serious limitations (Appendix 4, Table A3). With respect to the 16 cost- 
effectiveness analyses of iCBT for anxiety disorders, one study had minor limitations, 12 studies 
had potentially serious limitations, and three had very serious limitations (Appendix 4, Table 
A5). Eligible psychological interventions (e.g., individual or group face-to-face CBT) were not 
compared against each other in most studies, which led to potential overestimation of the QALY 
associated with guided CBT, particularly in patients with depression. Also, the additional value 
of iCBT over usual care was difficult to estimate given that: 1) study populations were exposed 
to prior treatments (medications or psychotherapies), and 2) sample sizes were relatively small 
and selected. 


In all studies, iCBT was provided over a short time frame (usually 3 months or less); as such, 
long-term extrapolations of its effect contain substantial uncertainty around the estimate of the 
ICER. Internet-delivered CBT had a larger QALY benefit when it was combined with usual care 
or provided within a collaborative care program. This creates the possibility of contamination of 
iCBT treatment effects when there is a cointervention with other case management approaches. 


A few large studies had a substantial amount of missing data that biased the estimation of the 
ICER. Also, direct medical costs associated with iCBT and management of depression or 
anxiety were not estimated in some studies. Studies that conducted probabilistic sensitivity 
analyses did not explore in detail changes in some important input parameters (e.g., changes in 
utility values or effectiveness of the iCBT intervetions) that could affect the robustness of cost- 
effectiveness results. Modeling studies that examined the cost-effectiveness of guided iCBT 
over 5 years, or over the participants’ lifetimes, made relatively bold assumptions regarding the 
constancy and equality of the treatment effect over time between face-to-face CBT and iCBT. 
These assumptions were without strong support in the evidence. 


Discussion 


Our review of the economic evidence suggests that guided iCBT probably represents good 
value for money for the management of adults with mild to moderate major depression or 
anxiety in the short-term (< 3 months). There is larger uncertainty around the economic 
estimates for people with mild to moderate major depression compared with anxiety disorders. 
Limited evidence from studies in patients with major depression suggests that unguided iCBT is 
as attractive as usual care and that guided iCBT provides a slight increase in QALYs when 
compared with unguided iCBT. 


No study examined a repetitive use of iCBT for the treatment of recurrent episodes of major 
depression and anxiety despite the fact that these diseases are episodic and recurrent. This 
observation is in line with the current recommendations that are positioning iCBT as a first-step 
in a stepped-care treatment model.'” 
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Some patients may prefer iCBT over face-to face CBT because it can be more compatible with 
their busy schedules or because they have a social phobia or other anxiety disorder. The greater 
privacy of iCBT may also reduce concerns around the stigma associated with mental illness.® 


Given the similar course of both major depression and anxiety disorders, our economic 
evaluation aimed to determine whether guided or unguided iCBT represents good value for 
money over a short-term horizon and over a lifetime. More specifically, we examined the cost- 
effectiveness of guided versus unguided iCBT against all available treatment alternatives that 
are used in Ontario for the initial management of mild to moderate major depression or anxiety 
disorders. Finally, we explored the lifetime cost-effectiveness of a stepped-care approach over 
usual care, starting with guided iCBT in adults with mild to moderate major depression and 
anxiety disorders. 


Conclusions 


Our evidence synthesis of the economic studies revealed that when compared with usual care, 
guided iCBT probably represents good value for money for the short-term management of mild 
to moderate major depression or anxiety disorders (e.g., panic disorder, social phobia, 
generalized anxiety disorder, and health anxiety disorder). It is unclear whether unguided iCBT 
is economically attractive in the treatment of adults with mild to moderate major depression and 
anxiety disorders. Given limitations of the model-based studies with respect to the extrapolation 
of the treatment effect of iCBT long term, the lifetime cost-effectiveness of guided iCBT remains 
unclear and warrants further investigation in the Ontario context. 
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PRIMARY ECONOMIC EVALUATION 


Our literature search identified 40 potentially relevant studies but none directly applicable to our 
research questions. Therefore, we conducted a primary economic evaluation. 


Research Questions 
Primary Research Questions 


e What is the cost-effectiveness of unguided or guided iCBT compared with face-to-face 
CBT or usual care in the management of adults with mild to moderate major depression? 


e What is the cost-effectiveness of unguided or guided iCBT compared with face-to-face 
CBT or usual care in the management of adults with anxiety disorders? 


Secondary Research Questions 


e What is the cost-effectiveness of guided iCBT within a stepped-care model, including 
face-to-face CBT, compared with usual care in the management of adults with mild to 
moderate major depression or anxiety disorders? 


e What is the cost-effectiveness of unguided iCBT followed by guided iCBT compared with 
usual care in the management of adults with mild to moderate major depression or 
anxiety disorders? 


Methods 


The information presented in this report follows the reporting standards set out by the 
Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement.®°” The 
methodological approaches follow the recent recommendations set out by the Canadian Agency 
for Drugs and Technologies in Health (CADTH) Guidelines for the Economic Evaluation of 
Health Technologies, fourth edition,®* and align with Health Quality Ontario’s Health Technology 
Assessments Methods and Process Guide. 


Type of Analysis 


For each research question, we conducted cost-utility analyses to estimate ICERs for the 
interventions versus comparator strategies. In these analyses, the ICER estimates are 
expressed as the cost per QALY gained. 


Target Population 


Our two study populations consist of individuals 18 years or older with a primary diagnosis of 
mild to moderate major depression or with an anxiety disorder as defined by the DSM-5 
criteria.°° In addition, the baseline severity of the disease is determined by the score on the 
nine-item Patient Health Questionnaire (PHQ-9),”° which also corresponds to the diagnostic 
criteria of the DSM-5.’"-”4 Individuals with mild and moderate major depression account for 
approximately 80% of all individuals with major depression.’”*:” The DSM-5° includes adults 
with the following anxiety disorders: specific phobias, social anxiety disorder (social phobia), 
panic disorder, agoraphobia, and generalized anxiety disorder. Anxiety disorders can exist in 
isolation, but they coexist with another type of anxiety disorder in more than 70% of patients, 
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including about 50% who also have major depression. Since the treatment for these conditions 
is similar,’”” our reference case analyses considered individuals with either primary diagnosis of 
mild to moderate major depression or one primary anxiety disorder. 


Our cohort of individuals, ready and able to engage in iCBT, is treated with iCBT one time only 
during an active episode of the disease. Ability to engage in iCBT is defined as the ability to: 


e Speak and understand at least one language in which the program is available (most 
applications are in English and require understanding of the English language at a 
grade 10 level) 

e Work with computers and use the internet 


Internet-delivered CBT is not suitable for people with severe learning disabilities, people with 
high risk of suicide, or people primarily diagnosed with a substance-related or addictive 
disorder.'* A generic program of iCBT tailored for mild to moderate major depression or anxiety 
disorders is unsuitable for people who are actively psychotic or manic; this program would need 
to be modified for people whose principal diagnosis is OCD, PTSD, an eating disorder, 
substance use, pain, or a somatoform disorder and would require a rigorous initial assessment 
(P. Farvolden, PhD, email communication, November 9, 2018). Our study also did not consider 
people with chronic conditions (i.e., cardiovascular disease, chronic pain, or diabetes) that 
caused or contributed to their depression or anxiety. We also did not consider populations with 
dementia or other cognitive impairment. Due to legal issues related to consent and 
confidentiality, there might be challenges in offering guided iCBT to individuals younger than 18 
(P. Farvolden, PhD, personal communication, January 19, 2018). 


Perspective 


We conducted the reference case analysis from the perspective of the Ontario Ministry of Health 
and Long-Term Care. We considered a societal perspective in a scenario analysis. 


Interventions and Comparators 


In the reference case analysis, we compared unguided and guided iCBT interventions with face- 
to-face CBT or usual care. Based on expert consultation and published literature,’*®° most 
participants self-refer (i.e., refer themselves) to treatment. Self-referral was the main method of 
accessing iCBT in the reference case analysis. 


Treatment of depression and anxiety disorders is provided in a stepped-care approach, 
depending on the disease severity, using low-intensity treatments such as unguided (stand- 
alone) or guided iCBT as first treatment options for people with mild to moderate depression or 
anxiety.'? Our secondary research questions included an examination of the stepped-care 
approach. The rationale for comparing face-to-face CBT to iCBT options in the reference case 
analysis was based on the NICE recommendations, patient preferences, and possible 
equivalency between various CBT treatments as summarized here. 


e NICE recommends using either iCBT (an individual-based treatment) or group face- 
to-face CBT as equally valid options for step 2 of the stepped-care approach in 
patients who present with mild to moderate depression or anxiety.'” Of note, step 1 
of the NICE model does not involve any psychological therapy, but active 
surveillance and education only. 
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e Some research suggests a possible equivalency in the effectiveness of in-person 
group CBT and guided iCBT for milder forms of depression.2°.°1 82 


In the following sections, we provide examples of iCBT programs and more detailed 
descriptions of each strategy. 


Internet-Delivered CBT 


Internet-delivered CBT is CBT delivered via the internet by an individual or program remote from 
the client. Table 6 summarizes various iCBT applications in Ontario and across the world. 


Table 6: Internet-Delivered CBT Models Used in Clinical Practice 


Type of iCBT Description 

Unguided iCBT Patients are provided information about website and participate in an online self-help 
program. 

Guided iCBT Patients undergo self-directed exercises with minimal/some guidance or coaching on 


how to use the website services. Generally, guides are peers and are not regulated 
health professionals (coaches) 


Patients complete iCBT modules and regularly discuss their progress with a regulated 
health professional therapist (e.g., who has a master’s degree in social work or 
psychology). 


Abbreviation: iCBT, internet-delivered cognitive behavioural therapy. 


Unguided iCBT 


Unguided iCBT is operationalized without any therapist support. Patients are told about the 
website and they independently guide themselves through an online CBT program. Some 
commercially available unguided programs offer minimal, technical support via help-line. 


Guided iCBT 


A guided iCBT intervention is operationalized as a structured psychological treatment delivered 
to a patient through the internet. The patient works through a course more or less 
independently, but with some kind of support from a therapist or trained coach.'*®*®° Based on 
the level of assistance and involvement provided to the patient, guided iCBT is divided into two 
categories, lower intensity coaching, and higher intensity therapist guidance. 


Coach-Guided iCBT 


Coaches are unregulated mental health workers trained in CBT. Their involvement in supporting 
and teaching iCBT is minimal. For example, a coach-guided iCBT program, Improving Access 
for Psychological Therapies (IAPT), is funded by National Health Service (UK).78:°°°87 It 
involves a five-step approach to psychological care for people with depression and anxiety. 
First, mental health services match treatment intensity to patient needs. ”°°°%°8” Step 2 allows 
access to low-intensity psychological support to patients presenting with mild to moderate 
symptoms of depression and anxiety.”*°°°°®” These interventions are often delivered by 
psychological well-being practitioners, who are predominantly psychologists with further training 
in delivering low-intensity CBT-based interventions.’®°°*°.8” Coaches in the IAPT program have 
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a university degree and are rigorously trained in CBT (S. Rao, MD, written and oral 
communications, January 25 2018). They are unregulated (nonregistered) clinicians or 
unregulated mental health workers whose main role is providing support for iCBT (with low 
contact with patients). They monitor and coach patients through the iCBT with the goal of 
providing structured CBT via internet modules so that the therapy is not influenced during in- 
person interactions. Coaches do not provide feedback on homework and do not text patients 
during the therapy. Their involvement is measured weekly, about 10 to 15 minutes per session. 
They may also perform many other administrative tasks during a regular work day. Training of 
coaches varies from 1 to 2 days to 1 year, but they work under the supervision of licensed 
health care professionals (registered experienced psychotherapists) and are supervised on a 
weekly basis. 


Therapist-Guided iCBT 


Therapist-guided iCBT is provided by regulated health care nonphysicians who are well-trained 
and certified in delivery of CBT. They spend more time with patients as compared to coaches. 
They monitor patient outcomes over time and communicate with patients, including regular 
check-ins and feedback on homework. 


Face-to-Face CBT 


Face-to-face CBT for major depression and anxiety is a structured, intensive, time-limited, 
symptom focused psychotherapy built on the premise that distorted beliefs about the self, the 
world, and the future maintain depressive affect. Once patients learn to recognize these 
automatic thoughts, they are taught more adaptive ways of responding.’''”3 


We previously conducted a cost-effectiveness analysis focused on the structured face-to-face 
approach to CBT, delivered through either individual or group therapy by a publicly funded, 
regulated (registered and professionally trained) health care professional.’” Face-to-face CBT 
was provided in 14 weekly sessions lasting an hour if provided by an individual therapist, or 

2 hours if provided as group therapy.’ The training of nonphysician therapists is regulated by 
professional organizations in Ontario. 


Usual Care 


Treatments for major depression and anxiety disorders are similar.’”” Usual care depends on the 
severity of disease. Active surveillance and education are recommended for mild major 
depression (no medication); pharmacotherapy may be recommended for moderate major 
depression and anxiety.’ 


Summary of Strategies Considered in Our Economic Analysis 


Table 7 summarizes all strategies evaluated in the economic model. The intervention and usual- 
care strategies did not depend on the severity of the disease, given the mix of disease severity 
in populations in whom iCBT is provided. Pharmacotherapy was initiated with the lowest daily 
dose of medication and in accordance with the 2016 Canadian Network for Mood and Anxiety 
Treatments (CANMAT) guidelines’'* and clinical practice (as established through expert 
consultation, and analysis’’). We describe the variability of medication pathways examined in 
the reference case and scenario analyses and the costs associated with additional care in 
Tables 11 and 12 and in Costs of Usual Care and Follow-Up, below. 
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As previously mentioned, unguided iCBT is delivered via the internet and does not involve any 
therapist support. Guided iCBT could be provided either by therapists or coaches. Our reference 
case analysis assumed that guided iCBT was provided by therapists (i.e., nonphysicians, 
regulated health care professionals) from Ontario, trained and certified to deliver CBT for 
depression and anxiety. In sensitivity analysis, we assumed that guided iCBT was provided by 
coaches, non-regulated mental health care workers, trained for delivery of CBT (i.e., the 
provision of CBT similarly to the UK IAPT program’®:®°°°87), Further explanations related to 
differences between regulated registered therapists and non-regulated mental health workers 
are described in the sections related to modeling main assumptions and cost input parameters. 


Table 7: Interventions and Comparators Evaluated in the Primary Economic Model in 
Individuals With Mild to Moderate Major Depression or Anxiety Disorders 
Interventions/Comparators 


1. Unguided individual iCBT 
e iCBT, 6-8 weekly sessions (modules) 


e iCBT, 6-8 weekly modules plus medication® 

2. Guided individual iCBT (with therapist) 

e CBT, 8-10 weekly sessions (modules) 

e CBT, 8-10 weekly sessions plus medication? 

3. Face-to-face individual CBT by nonphysician 

e CBT, 14/10° 1-hr weekly sessions 

e CBT, 14/10° 1-hr weekly sessions plus medication? 
4. Face-to-face group CBT by nonphysician 

e CBT, 14/10° 2-hr weekly sessions 

e CBT, 14/10° 2-hr weekly sessions plus medication? 
5. Usual care 

e No CBT, no treatment 


e Medication? 


Abbreviation: CBT, cognitive behavioural therapy. 

*Example of medication used in our analysis is sertraline starting at an initial dose of 50 mg/d. 

14 in-person sessions of CBT for major depression and 10 in-person sessions of CBT for an anxiety disorder, with 2 additional sessions, one for the 
assessment and the last for planning and conclusion/relapse prevention.’” 


Discounting and Time Horizon 


Internet-delivered CBT is a first step in the management of depression and anxiety and is not 
repeated in individuals who subsequently sustain a relapse or recurrence. Therefore, in the 
reference case analysis, we used a short time horizon of 1 year for the duration of iCBT, or CBT 
and follow-up. No discount rate is applied to accumulated costs and QALYs. This follow-up time 
is based on the data provided by most studies examining the effectiveness of iCBT. It is further 
confirmed through our expert consultations. 


We explored the cost-effectiveness of guided iCBT for two different stepped-care models with 
different time horizons: 


e Inthe first analysis, we examined the cost-effectiveness of guided iCBT with a therapist 
followed by face-to-face CBT and compared it with usual care within a long-term 
stepped-care model. The time horizon for this analysis was lifetime; consequently, we 


Ontario Health Technology Assessment Series; Vol. 19: No. 6, pp. 1-199, February 2019 73 


Primary Economic Evaluation February 2019 


applied a discounting rate of 1.5% for both costs and QALYs, as suggested by the 
CADTH Guidelines for the Economic Evaluation of Health Technologies.® 

e Inthe second analysis, we examined the cost-effectiveness of unguided iCBT followed 
by guided iCBT with a therapist solely in individuals who are likely to drop out. This 
model had a short 1-year time horizon with no discounting applied to the cost- 
effectiveness outcomes. 


Model Structure: Reference Case Analysis 
Reference Case Analysis: Primary Research Questions 


We developed a decision-tree model to determine the incremental cost per QALY gained of the 
iCBT strategies versus usual care or face-to-face CBT for one episode of mild to moderate 
major depression or an anxiety disorder. Due to limited evidence with respect to iCBT, this 
short-term model does not account for repetitive use of iCBT in a subsequent recurrence of the 
disease,” or the risk of hospitalizations. This decision was supported by our economic 
evidence review, in which most studies provided a single course of iCBT (6-10 weeks on 
average). The disease progression and repetitive use of CBT should be addressed within the 
stepped-care treatment approach with the use of face-to-face CBT or other types of 
psychotherapies.?>° In our prior analysis of structured psychotherapies for major depression 
and generalized anxiety disorder in response to which the Ontario Health Technology Advisory 
Committee made a positive funding recommendation with respect to structured 
psychotherapies, we examined the long-term economic impact of repetitive use of group and 
individual face-to-face CBT for populations with major depression and anxiety.”” 


Since iCBT and face-to-face CBT interventions are provided using similar methodology across 
major depression and anxiety disorders, and their effectiveness is measured using the same 
outcomes, we used the same model structure to address our primary research questions. It is 
worth noting that although anxiety disorders can exist in isolation, they coexist with major 
depression in at least half of the cases.'”**°° Therefore, the definition of target population was 
based on the primary diagnosis (e.g., major depression). Also, a Subgroup analysis 
differentiating health-related utility weights between mild and moderate stages of the disease 
may not be plausible, as our economic evidence review found that unguided and guided iCBT 
were provided in the mixed sets of populations in terms of baseline disease severity. 


Figure 3 represents an example of the model schematic used to evaluate the cost-effectiveness 
of iCBT over control treatments in adults with mild to moderate major depression. As mentioned 
above, the model structure is similar for the two primary research questions. The differences are 
the smaller number of face-to-face sessions used for the treatment of anxiety disorders and 
values of some input parameters. At the beginning of the simulation, adults start with mild to 
moderate major depression, with a possibility of medication for those with a more severe 
disease (i.e., about 50% of patients with moderate major depression, accounted only through 
modeling of the treatment costs). The model accounts for the probability of participant drop out 
after each intervention and assumes that 25% to 30% of patients recover spontaneously after 
discontinuation of the therapy.’ In the next step, if patients do not respond to CBT (or iCBT) 
and progress with the disease, a medication is added to the clinical pathway (our model 
assumes about 50% of participants receive medication; costs of medications are included in our 
model. More details are provided in later sections related to the costing of usual care. The 
outcomes associated with the effectiveness of the interventions within this short time horizon 
are response (i.e., 50% reduction in depression scales during the first weeks of therapy 
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initiation) and remission/recovery (i.e., no new episode of depression or no relapse within the 
3-month continuation phase of treatment).>° 


Respond and recover 


Medication pathway 


Continue | 4 
Recover 
| Recovery 

| Do not respond | Medication 

<AT/CAT { ’) 
ic | [No recovery 
nteryention | Sieeeeeaeeee 


| Donot recover 
Recover Recovery 


|No medication 

| [No recovery eet 
Adults with | Drop out ee ~<| 
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major depression _ | 
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Figure 3: Simplified Model Schematic: Reference Case Analysis 


Abbreviation: iCBT, internet-delivered cognitive behavioural therapy. 

Note: Figure 3 describes a decision-tree model where the square denotes a decision node, the circle denotes a chance node and the triangle denotes 
a terminal node (a health state that accumulates utilities and costs associated with a specific pathway and strategy, assuming a 12-month time 
horizon). The same model structure was applied to the reference case analysis in adults with an anxiety disorder. A person with mild to moderate major 
depression who enters this model chooses between five different strategies: unguided iCBT, guided iCBT, individual face-to-face CBT, group face-to- 
face CBT (these four tracks compose the upper branch), and usual care. Within each strategy, pathways account for differences in drop out, response 
and recovery rates, utilities, and costs. People who respond and recover enter the well health state. People who do not respond are offered 
medication. Those who recover progress to the better health state associated with increments in health-state utility, and those who do not recover do 
not accumulate this utility increment (i.e., return to the baseline health state). 


Main Assumptions: Reference Case Analysis 


The major assumptions for this model are as follows: 


e Each individual in the iCBT intervention received only one course of treatment, 
regardless of disease progression (one-time non-repetitive treatment) 

e No stepping-up the care—those who begin with medication remain on their medication 
throughout the course and those receiving iCBT do not switch to an in-person 
psychological therapy 

e In adults who are taking medication, an increment in the costs was accounted for, but 
not an increment in utility. The effect of medication on quality of life or clinical outcomes 
was not separated from the effects of other treatments (see our economic evidence 
review) 

e In adults started with CBT or iCBT only, medication was added after no response to CBT 
treatment. In a scenario analysis, we explored the addition of medication from the 
beginning of the iCBT/CBT intervention and the change in medication pathway after no 
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response to CBT/iCBT treatment (more details provided in Table 12 and Costs of Usual 
Care and Follow-Up, below) 

e Booster iCBT sessions (i.e., an additional course of treatment) were not accounted for 
the following reasons: 


oO 
1e) 


They require the full fee for iCBT 

They suggest deterioration or reappearance of symptoms (recurrence of the 
disease) 

They require modeling over a longer time horizon 

There is limited evidence of the efficacy of repeated iCBT on the risk of relapse 
or recurrence 

There is limited evidence of the long-term effects of iCBT on the risk of 
recurrence of major depression and anxiety or other health outcomes such as 
rate of suicide or hospitalization 


e Probability of suicide and excess mortality were assumed to be small in patients with 
mild forms of disease and was not accounted in the short-term model. Psychotherapies 
were not suitable for people with severe major depression or at risk of suicide’’ 

e Assumptions related to program delivery and costs of treatment: 


O 


The number of sessions and costs of providing iCBT for anxiety disorders or 
major depression were assumed to be similar (a total of eight therapist-guided 
sessions over 8 weeks) 

The cost of iCBT was incurred with the program initiation; there was no reduction 
of cost from people who dropout 

Training costs for iCBT: 

« High-intensity guided iCBT included the training costs in therapist salaries 
(expert consultation); 

« lf iCBT was guided by coaches, then the training cost would not be 
accounted in their salaries, but would be based on findings from the 
literature data®° because training costs are treated as proprietary 
information (P. Farvolden, PhD, email communication, March 2018) 

Based on our prior HTA of structured psychotherapies for major depression and 
generalized anxiety disorder (GAD)’’, the face-to-face CBT strategy incurred 
costs for psychological assessment with referral 

Self-referral was assumed for all types of iCBT interventions, except for the 
coach-based program (described in Total Costs: Unguided or Guided iCBT, 
below) 

We assumed that only patients who had been prescribed medications have 
regular follow-up visits with their physicians” 

Internet-delivered CBT was assumed to be provided via a central portal (e.g., 
Ontario Telemedicine Network [OTN]) to account for the maintenance costs 
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Secondary Exploratory Analyses: Model Structures 


Long-Term Cost-Effectiveness of Guided iCBT Provided Within a 
Stepped-Care Model 


We combined our short-term reference case iCBT model with a previously developed 
probabilistic Markov microsimulation model’’ to examine the lifetime cost-effectiveness of 
guided iCBT with a therapist and face-to-face CBT versus usual care in people with new and 
recurrent major depression and anxiety. We used a stepped-care approach starting with 
therapist-guided iCBT followed by with group or individual face-to-face CBT if the disease recurs 
or progresses. In this analysis, we compared three strategies: 


e Guided iCBT followed by individual CBT provided by a regulated health care 
professional (nonphysician) 

e Guided iCBT followed by group CBT provided by a regulated health care professional 
(nonphysician) 

e Usual care (no CBT) 


The methods including model input parameters are described in detail in Appendix 5. 


Cost-Effectiveness of Unguided iCBT Followed by Guided iCBT 


Based on the literature'>*' and expert consultation (J. Hensel, MD, oral communication, 
February 7, 2018), unguided iCBT could be used as a first step in stepped care. It could be 
used to select and help those with minor issues and identify others who need more help with 
CBT assignments, are not motivated enough to self-manage their CBT, or are at risk of early 
dropout. Alternatively, as highlighted with the UK Improving Access for Psychological Therapies 
program (IAPT) and the literature, using a stepped-care approach with counselling/education as 
step 1 and iCBT as step 2 may increase access to mental health services and treatment. '7:78-®° 


We developed a stepped-care decision-tree model comparing the costs and effectiveness of 
unguided and guided iCBT modeled as a stepped-care strategy versus treatment as usual over 
1 year. This scenario did not include face-to-face CBT because this type of CBT is part of 
stepping-up the care from iCBT to high-intensity structured in-person psychotherapy for those 
who do not respond to prior treatments or experience a relapse or recurrence (step 3 under 
NICE guidelines’). 


Clinical Outcomes and Utility Parameters: Reference Case Analysis 


We used several different input parameters to populate the model. These parameters describe 
the course of one major depression or anxiety episode, including changes in the severity of 
initial disease with the treatment (e.g., from mild to moderate major depression). To estimate the 
cost-effectiveness of iCBT versus usual care or face-to-face CBT, we populated the model with 
parameters related to the effectiveness of interventions (e.g., probabilities of response and 
recovery), health state utilities, and direct medical costs. 


Natural History 


To model the natural history of one episode of major depression, we used literature sources and 
data from current clinical practice provided through expert consultation (Table 8). Additional 
input parameters for the life-time cost-effectiveness analysis (Secondary research questions) 
are described in detail in our prior analysis of structured psychotherapies for major depression 
and GAD and Appendix 6.”” 
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The probabilities of dropout from face-to-face and iCBT are based on a meta-analysis 
conducted by van Ballegooijen et al,°? which included 24 studies with 981 participants allocated 
to face-to-face or guided iCBT. This study was not included in our clinical review but is 
described here to address potentially high drop out in unguided iCBT compared with the rates of 
drop out from guided iCBT and face-to-face CBT. The probability of adding a medication after 
failing to respond to CBT is based on meta-analyses conducted within the most recent UK 
modelling analysis in people with moderate major depression.% After dropout, major depression 
could resolve on its own or become more severe. We assumed that a person has a 25% chance 
of recovery without any treatment after a dropout.” 


Table 8: Natural History Inputs, Reference Case Analysis: Major Depression and 
Anxiety Disorders 


Model Parameter Mean Source 
Probability of dropout, major van Ballegooijen et al, 2014°? 
depression/anxiety 

Face-to-face CBT 0.16 

Unguided iCBT 0.32/0.754 

Guided iCBT 0.19 
Probability of dropout, major Machado et al, 2006% 
depression/anxiety 

Sertraline 0.10 

All SSRIs 0.28 (0.19-0.38) 
Probability of self-recovery after dropout 0.25 Guidelines, expert opinion’”” 
(no treatment) 
Probability of adding medication Koeser et al, 2015°° 

iCBT/CBT 0.23 

Usual care 0.70 


Abbreviations: CBT, cognitive behavioural disorder; iCBT, internet-delivered CBT; Cl, confidence interval; SSRI, selective serotonin 
reuptake inhibitor. 
*Values used in sensitivity analysis. 


Intervention Effects 


The clinical evidence review of this report provides data related to the effectiveness of guided 
iCBT as compared to usual care.”7°.21-235° Data related to the effectiveness of unguided iCBT 
was obtained from a systematic review by Karyotaki et al®' (this study that was excluded from 
the clinical review because of the lack of a precise definition of the target population with major 
depression). We assumed that the recovery rate for guided iCBT is the same for these two 
disorders, and as reported in a systematic review by Andrews et al.?' In their review, the authors 
reported the effectiveness estimate for the follow-up of 9-18 months for people with major 
depression and anxiety disorders. The effectiveness of face-to-face CBT versus usual care was 
based on data from our prior report.’” The effectiveness of these therapies was measured using 
a Hedge’s g statistic or standardized mean difference (see column 3 in Tables 9 and 10). Below, 
we describe the probabilities of response and recovery calculated from the identified effect sizes 
using the recommended effect-size conversion method.® Based on data from three systematic 
reviews,”2'-3 the probability of response to guided iCBT was assumed to be equal to face-to- 
face CBT. This assumption was based on data from Dedert et al,?° Andrews et al,?' and 
Adelman et al,’ that reported no statistically significant difference in the response between 
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guided iCBT and face-to-face CBT. Of note, Dedert et al warned that despite no difference 
between two types of treatments, the evidence on equality of face-to-face CBT and guided iCBT 
is very limited, particularly for adults with major depression. Although we made our assumption 
of no difference in responses to guided iCBT and face-to-face CBT in our target populations, we 
distinguished the effects of guided iCBT and face-to-face CBT on recovery. Therefore, we 
remained in line with the overall conclusions of our clinical review. 


Table 9: Effectiveness of Interventions, Reference Case Analysis: Mild to Moderate 
Major Depression 


Model Parameter Mean (95%CI/SE)@ SMD (95%Cl)* Source 
Probability of response (improvement) 

Unguided iCBT 0.50 (0.28—0.62) 0.27 (0.17—0.37) Karyotaki, 201 7°! 
Guided iCBT 0.73 (0.66-0.77) 0.83 (0.59-1.07) Arnberg et al, 20142° 
Individual face-to-face CBT 0.73 (0.66—0.77) NA Dedert et al, 201723; 


Andrews et al, 20182"; 
Adelman et al, 2014’ 


Group face-to-face CBT 0.73 (0.66—0.77) NA Dedert et al, 201723; 
Andrews et al, 20187'; 
Adelman et al, 2014’ 


Medication 0.70 Koeser et al, 2015°8 


Probability of recovery 


Unguided iCBT 0.38 (0.09) NA Kaltenthaler et al, 2006°° 

Guided iCBT 0.48 (0.23—-0.60) 0.22 (0.59-1.07) Andrews et al, 20187" 

Individual face-to-face CBT 0.66 (0.64—0.71) NAS Biesheuvel-Leliefeld, 
2016% 

Group face-to-face CBT 0.63 (0.59-0.66) NA° Churchill, 20019” 

Medication 0.62 (0.58—-0.66) NAS Cipriani, 2014% 


Abbreviations: CBT, cognitive behavioural therapy, Cl, confidence interval; iCBT, internet-delivered CBT; NA, not applicable; SE, standard error; SMD, 
standardized mean difference. 

4Original value, reported in the article and clinical review: probability of response/recovery was calculated by transforming SMD to log odds ratio using 
the formula: d x t1/sqrt(3),°° and further calculated the probability of event based on the data of baseline probability of response.®° 

Beta distributions assigned in probabilistic sensitivity analysis to all parameters for which 95% Cl or SE were specified in Table 9a. Two parameters of 
the beta distribution (a, B) are derived from the mean and SE (stated for each model parameter). Formulas for these calculations, derived from the 
mean and SE (provided in the Tree Age software) are a = ([Mean?] x [1 — Mean])/([SE?] — Mean); B = ([{1 — Mean} x {1 — Mean}] x Mean)/([SE?] — 1). 
°Reported risk of recurrence, in-person CBT vs. usual care in Biesheuvel-Leliefeld, 2016 (relative risk: 0.68, 95% Cl 0.53—0.76) and group CBT, in 
Churchill, 2011 (0.78, 95% Cl 0.69-0.93), and sertraline in Cipriani, 2014 (relative risk: 0.80, 95% Cl 0.69-0.93) details provided in our prior report.’” 
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Table 10: Effectiveness of Cognitive Behavioural Therapy Interventions, Reference Case Analysis: 
Anxiety Disorders 
Model Parameter 


Mean (95%CI/SE)? SMD (95%Cl) Source 


Probability of response (improvement) 


Unguided iCBT 
Guided iCBT 


Individual face-to-face CBT 


Group face-to-face CBT 


Medication 


0.67 (0.58-0.73) 
0.74 (0.69-0.78) 
0.73 (0.66-0.77) 


0.73 (0.66-0.77) 


0.70 


0.66 (0.32-1.00) 
0.84 (0.45-1.23) 
NA 


NA 


Adelman, 20147 
Arnberg et al, 20142° 


Dedert et al, 201723; 
Andrews et al, 20187"; 
Adelman et al, 2014’ 


Dedert et al, 201723: 
Andrews et al, 201871: 
Adelman et al, 20147 


Koeser et al, 2015%3 


Probability of recovery 


Unguided iCBT 
Guided iCBT 


Individual face-to-face CBT 


Group face-to-face CBT 


Medication 


0.38 (0.09) 
0.48 (0.23-0.60) 
0.66 (0.62-0.68) 


0.63 (0.59-0.66) 
0.62 (0.58—0.66) 


NA 
0.22 (0.59-1.07) 
NAC 


NAS 
NAS 


Kaltenthaler et al, 20065° 


Andrews et al, 2018?! 


Biesheuvel-Leliefeld, 
2016% 


Churchill et al, 20019” 
Cipriani et al, 2014°8 


Abbreviations: CBT, cognitive behavioural disorder; Cl, confidence interval; iCBT, internet-delivered CBT; NA, not applicable; SMD, standardized 


mean difference. 


4Beta distributions assigned in probabilistic sensitivity analysis to all parameters for which 95% Cl or SE were specified in Table 9b. Two parameters 
of the beta distribution (a, 8) are derived from the mean and SE (stated for each model parameter). Formulas for these calculations, derived from the 
mean and SE (provided in the Tree Age software) are a = ([Mean?] x [1 — Mean])/([SE?] - Mean); B = ([{1 — Mean} x {1 — Mean}] x Mean)/([SE?] - 1). 
Original value, reported in the article and clinical review. 

°Reported risk of recurrence, in-person CBT vs. usual care in Biesheuvel-Leliefeld, 2016 and group CBT, in Churchill, 2011, and sertraline in 
Cipriani, 2014, details provided in our prior report.’” 


Health State Utilities 


A health state utility is a measure of health-related quality of life and reflects the strength of 
preference for specified health states. By convention, health state utilities are anchored on 
death and best possible health (death is assigned a utility weight of 0, and perfect health is 
assigned a utility weight of 1).9° The value of a QALY for a certain health state is calculated by 
multiplying time spent in that health state with the utility assigned to that health state 

(e.g., 1 year of untreated schizophrenia with a utility weight of 0.5 equals a QALY of 0.5). We 
performed a targeted literature search for utility values on February 28, 2018, for studies 
published from inception to the search date in MEDLINE. The search was based on the 
intervention of the clinical search strategy with a methodological filter applied to limit retrieval to 
health state utility values.'°° See Appendix 1 for literature search strategies, including all search 
terms. We also used the utility weights specific to disease states that were reported in economic 
evaluations examined in our economic review of the literature. 


The utility weights related to health states of depression and anxiety disorders considered for 
Our analysis are presented in Table 11. We present the original values found in the literature 
and the model input parameter values estimated using the area-under-the-curve method to 
account for changes in the utility estimate over time.'°' For example, in Table 11, the utility 
value for guided iCBT for anxiety disorders was estimated for the follow-up period of 12 months 
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using the following formula: ([{0.77 + 0.82}/2] - [1/3]) + ({{0.82 + 0.85}/2] - [1/6]) + 
([{0.85+0.85}/2] - [1/2]) = 0.265 + 0.139 + 0.465 = 0.83. 


Utility weights associated with medication therapy are obtained from a study including Canadian 
patients with past mild or moderate major depressive episodes. '”* The face-to-face CBT-related 
utility value is based on a UK study of 457 adults with major depression and generalized anxiety 
disorder who received psychological therapy.'°? Based on studies found in our economic 
literature review and targeted search, compared with treatment as usual, the increments in 
utilities are nonsignificant with unguided iCBT compared with usual care,** ranging from 0.01 to 
0.05 with guided iCBT.%°°*'™ In studies that examined iCBT, utility weights were elicited using 
the EQ-5D instrument. 


People who are “well” are assumed to be stable and not receiving treatment. Lenert et al'® 
conducted a mapping study using a sample of 295 people with major depression to provide the 
utility weights for those who are “near normal health.” 
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Table 11: Health State Utilities Considered for the Economic Model 


Model Parameter: Utilities Cee Cl)? Source 
Treated major depression Schaffer et al, 200210 

Mild MDE 0.79 (SD: 0.28) 

Moderate MDE 0.67 (SD: 0.36) 
Unguided iCBT, major 
depression/anxiety 

Unguided iCBT vs. usual care -0.01 (—0.06 to 0.09) Duarte, 2017,°” Littlewood, 2015%° 
Guided iCBT, major depression, at 0.84 (0.81 to 0.86) Duarte, 2017,°7 Littlewood, 2015%° 
12 mo 
Guided iCBT, anxiety Hedman et al, 201 158.104 

Baseline 0.77 (SD: 0.18) 

4mo 0.82 (SD: 0.14) 

6-12 mo 0.85 (SD: 0.14) 

Utility at 12 mo 0.83 (0.81 to 0.84) 
Individual face-to-face CBT, major King et al, 20001 
depression/anxiety 

Baseline 0.73 

4mo 0.85 (NR) 

12 mo 0.85 (NR) 

Utility at 12 mo 0.83° 
Group CBT, major Hedman et al, 201 158104 
depression/anxiety° 

Baseline, usual care 0.74 (SD: 0.19) 

4mo 0.80 (SD: 0.17) 

6-12 mo 0.81 (SD: 0.17) 

Utility at 12 mo 0.80 (0.77-0.82) 
Normal health, well health state 0.94 (SE: 0.03) Lenert et al (2000)! 


Abbreviations: CBT, cognitive behavioural therapy; Cl, confidence interval; iCBT, internet-delivered CBT; MDE, major depressive episode; 
NR, not reported; SD, standard deviation; SE, standard error. 


‘Beta distributions assigned in probabilistic sensitivity analysis (please see above formulas for calculations of the beta distribution’s parameters: 
a and B. 


’Standard error assumed to be the same as the one estimated for the group CBT. 
°Assumed to be the same for anxiety disorders and major depression, based on results of a study in patients with social anxiety disorder. 


Cost Parameters: Reference Case Analysis 


The reference case analysis included additional costs relevant to CBT treatment. Scenario 
analyses considered the costs of follow-up and costs borne by patients. 
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Overall Components of Cognitive Behavioural Therapy Treatment: Reference 
Case Analysis 


Depending on the type of CBT, the costs of treatment potentially include the following 
components: 


e Initial assessment, primarily applicable to face-to-face CBT 
o The visit to a general practitioner or nurse practitioner to obtain a referral; this 
is applicable to face-to-face CBT and one type of guided iCBT (i.e., coach- 
guided iCBT) 
o The psychiatric assessment by a psychiatrist or psychologist 
e Labour 
o Face-to-face CBT by regulated nonphysician therapists 
o Costs of supervision (1 hour weekly) 
e Internet-delivered CBT 
o Non-regulated mental health care workers for coach-guided (low intensity- 
guided) iCBT 
o Regulated nonphysician therapists (e.g., those with a Master’s of Social Work 
degree) for therapist-guided (high-intensity guided) iCBT 
e Physician follow-up, applicable to people taking medications (a description of specific 
medication pathways is presented in Table 16) 
e Maintenance and licencing specific to the delivery of iCBT 


Labour Costs 


We used a previously described method to estimate the labour costs associated with clinical 
activities. We also applied an hourly salary where appropriate.’” Applied cost recognizes that 
salaried clinicians spend time on nonpatient activities, so less than 100% of a clinician’s time 
accounts for clinical work.'°6 It is recognized that the applied rate (i.e., the ratio of applied time 
to total time) is 85% for medical staff. Using the applied rate, we calculated hourly costs for 
publicly funded regulated therapists providing face-to-face CBT.’’ The following is an example 
of these calculations that are further expanded to estimate the hourly rate for therapists who 
deliver guided iCBT (regulated professionals and coaches). 


Assumptions: 


e A full-time equivalent (FTE) works 1,950 hours per year (7.5 hours per day, 5 days per 
week, 52 weeks per year) 

e Using the applied rate of 85%, the applied time is thus 1,950 x 0.85 = 1,658 hours 
annually 


Costs: 


e Individual face-to-face CBT by regulated nonphysician therapist at an annual salary of 
$110,000 with 30% benefits ($143,000 per year total) yields an hourly cost of $86.25 
(14 weekly 1-hour sessions for major depression and 10 weekly sessions for anxiety 
disorders) 
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e Group face-to-face CBT including 10 participants by one therapist yields an hourly cost 
of $17.25 per participant (14 weekly 2-hour sessions for major depression; 10 weekly 
2-hour sessions for anxiety disorders) 

e Coach-based iCBT therapist at an annual salary of $35,000 with 30% benefits ($45,500 
per year total) yields an hourly cost of $27.44 

e Therapist-based iCBT by regulated MSW therapist at an annual salary of $54,455 with 
30% benefits ($70,791 per year total) yields an hourly cost of $42.70 (P. Farvolden, 
PhD, email communication, March 24, 2018) 

e Supervision for face-to-face CBT was $86.25 for each 1-hour weekly session by an 
experienced clinical psychologist (10 weeks for major depression and 8 weeks for 
anxiety disorders). Supervision costs do not include the first and last week of CBT, which 
were primarily related to treatment planning 

e Overall costs of supervision for iCBT (not per patient) consisted of 1-hour weekly 
supervision by a clinical psychologist ($86.25 per hour cost) over 8 weeks of iCBT for all 
but the coach-based program, which is provided for a maximum of six sessions 


Maintenance Costs: Internet-Delivered CBT 


The maintenance and IT support required for iCBT to be provided through a central portal may 
be supported by an annual government grant ranging from $50,000 to $100,000. It was 
assumed that people willing to engage in iCBT would have internet access. The capacity for 
guided iCBT may depend on the number of users and may range between 90 and 120 users 
per week (P. Farvolden, PhD, written communication, March to May 2018). Although the 
capacity for unguided iCBT over the online or research portal should be effectively unlimited, we 
assumed the same estimated cost of maintenance as for the guided iCBT. Table 12 presents 
the per-person estimates of maintenance costs for an iCBT course. The lower-range estimated 
cost of $77 was used for the reference case. 


Table 12: Estimates of Web Portal Maintenance Costs for iCBT 


Mean, $? 
Maintenance Costs ($) LowEnd High End Source 
Annual maintenance grant 50,000 100,000 — Consultation® 
Total cost per week (52 wk) 961 1,923 NA 
Cost per week per patient accounting for iCBTcapacity,100 9.61 19.23 NA 
patients® 
Total: 8-wk guided iCBT course 76.92 153.85 NA 


Abbreviation: iCBT, internet-delivered cognitive behavioural therapy. 

All costs were estimated in 2018 Canadian dollars. 

>Communication with experts, names and affiliations withheld to protect proprietary information. 

°Assumed capacity is the midpoint between 90 to 120 patients weekly (Applicant, email communication, March 24, 2018). 


Total Costs: Unguided or Guided iCBT 


Table 13 describes the estimates for per-person costs for different types of iCBT. Based on 
expert consultation and literature findings,'°’ the amount of time a therapist spends providing 
support for iCBT to a client is at least three times shorter than the time spent for face-to-face 
CBT. As previously mentioned, guided iCBT can involve a therapist (i.e., high intensity of 
communication between a client and the therapist) or a coach (i.e., low intensity of 
communication). A therapist (an example of high-intensity guidance) spends about 20 minutes 
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on each client per guided iCBT session. Based on the UK IAPT data, ®°°%°°” a coach spends 
half as much time as a highly qualified therapist (i.e., 10 minutes per client per session, written 
and oral communication, S Rao, MD, January 25, 2018, see description in intervention section). 
Applying previous calculations of their hourly cost, we estimate the labour costs associated with 
one 8-session coach-guided or therapist-guided iCBT course is $37 and $114, respectively 
(Table 13). The cost of supervision is about $690 per course (assuming 8 weeks of sessions). 


We also accounted for the cost of training of coaches based on data from an Australian 
research study by Crome et al.°° In this study, a 2-day highly intensive training workshop was 
about $579 CAD ($640 2014 AUD converted to 2018 CAD using purchasing power parity and 
consumer price index conversion factors. 


Scenario analyses include other models of coach- or therapist-based iCBT: 


e Scenario A considers coach-based iCBT as provided by one 6-session program in Ontario 
conducted in adults with mood disorders. This program is not an exemplar of structured 
coach-guided iCBT; therefore, it is explored in a scenario analysis for adults with major 
depression only. The probability of response is about 40% at the end of the program and the 
recovery rate is unknown. Consequently, we assumed that this program will have the same 
probability of recovery as that calculated for therapist-guided iCBT (mean 0.48, 95% Cl 
0.23—0.60, see Tables 9 and 10). 

e Scenario B represents an example of a possible primary care iCBT model whereby a family 
physician is trained to be the coach for iCBT and to overview the whole patient care. This 
iCBT model is important to consider as patient care in many rural and Northern Ontario 
locations are dependent on primary care physicians.’” This scenario could be the easiest 
and most reliable way for implementation in some regions of Ontario. An example of this 
type of model is rolling out in Newfoundland (in-person/e mail communication, Dr. P. 
Cornish, 01 Feb, 2018).'% 

e Scenario C describes one type of privately (employer) funded therapist-guided iCBT 
currently existing in Ontario. The employer provides a therapist-guided iCBT program 
through their own portal (not publicly funded) (P. Farvolden, PhD, multiple communications, 
January to March 2018). Depending on the employer’s capacity to provide care for people 
with major depression and anxiety, this could be also an implementation option to consider 
for Ontario. 


The reference case analysis used the mean costs estimated from the low range values (e.g., 
$137 unguided iCBT). The upper range value was used in sensitivity analyses to examine the 
robustness of cost effectiveness results to changes in the costs of iCBT. All scenario analyses 
were done from a conservative perspective, using the upper-level mean cost estimates (i.e., the 
mean cost for the scenario A: $774). All cost estimates were considered probabilistically in our 
analyses using the gamma distribution to account for uncertainty around the mean estimate. 
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Table 13: Cost Estimates for Unguided and Guided iCBT Courses 


Mean Costs, $2 


February 2019 


Total Cost of 


Licence Therapist Therapist 
Type of iCBT Course (Low-High) Referral Training Salary 


iCBT Course, $2 
Supervision Maintenance (Low-High)® 


Reference case analysis 


Unguided iCBT course 60-90 0° NA NA 
Therapist-guided iCBT 3-54 0° 579 114 
(high intensity) course 

via portal 

Therapist-guided iCBT 3-54 0° NA 114 
(high intensity) 

employer-led therapist 

course via portal 


Coach-guided iCBT 3-54 0° 579 37 
(low intensity) course 


NA 77 137-167 
690 77 1,466—1,470 


690 77 887-891 


690 77 1,389-1 393 


Scenario analyses 


A. Coach (low intensity) 60-90 62.75 NA 27 
guided iCBT course 


with referral® 
B. GP- and coach- 3-54 62.75 579° 167° 
guided iCBT' 
C. Employer-led, 500-800 0° NA NA 


publicly provided iCBT 
course 


517 77 744-774 


690 77 1,582—-1 ,587 


NA 77 577-877 


Abbreviations: iCBT, internet-delivered cognitive behavioural therapy, all assumed to have 8 sessions except for the Bounce Back program (6 sessions); NA, not applicable. 


4All costs were estimated in 2018 Canadian dollars. 


’Gamma distributions assigned in probabilistic sensitivity analysis, assuming standard error to the mean cost of 25%. Two parameters of the gamma distribution (a, A) are derived from the 


mean and standard error. Formulas for these calculations are: a = (Mean‘2)/(SE“2); A= Mean/([Mean x SE]2). 


°We assumed patient referred self and costs of assessment were included in price of the program (applicant, in-person communication January 19, 2018). 
4Costs provided by the applicant (email communication March 24, 2018, and updated in May 2018): these are the monthly costs ($3-$5 per person), the total therapist-guided iCBT assumed 


8 weekly sessions of therapy and were accounted in the estimates of the total per-person costs. 
°Costs estimated for the one example coach-based program in Ontario. 


‘Costs estimated for the primary care iCBT model, enabling a circle of care and assuming that the coach is a family physician (one session/one visit of $62.75). 
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Total Costs: Face-to-Face Cognitive Behavioural Therapy 


Table 14 describes the components, calculations, and estimates of total per-person costs for 
individual and group CBT in people with major depression or an anxiety disorder. The 
calculation methods for therapist cost (i.e., hourly cost per CBT session) and costs of 
supervision are described above. The costs of in-person CBT represent the conservative 
estimate. Actual costs may be lower. For instance, it is possible that supervision costs will not 
be needed for experienced-regulated CBT therapists. In sensitivity analyses, we used a lower 
range for costs associated with individual and group CBT (hourly cost estimates for individual 
face-to-face CBT of $77.60 and for group CBT of $15.50), a total cost amounting to $2,242 and 
$1,372, for major depression and $1,219 and $598, for anxiety disorders, respectively, for 
individual and group CBT. 


Table 14: Estimated Per-Person Cosis of Face-to-Face Cognitive Behavioural Therapy 
Course by Regulated Nonphysician? 


No. of Per-Person Cost, $? 
Sessions, 
Course/ Initial Therapist Weekly 
CBT Format Supervision Assessment Salary Supervision Total” 
Individual CBT, major 14/12 223.35 1,207.5 1,034.9 2,466 
depression 
Group CBT, major 14/12 223.35 241.5 1,034.9 1,500 
depression 
Individual CBT, anxiety 10/8 223.35 862.5 690 1,776 
Group CBT, anxiety 10/8 223.35 172.5 690 1,086 


Abbreviations: CBT, cognitive behavioural therapy. 
All costs were estimated in 2018 Canadian dollars. 


®Gamma distributions assigned in probabilistic sensitivity analysis, assuming SE to the mean cost of 25%. Two parameters of the gamma distribution 
(a, A) are derived from the mean and standard error. Formulas for these calculations are: a = (Mean?)/(SE*); A= Mean/([Mean x SE] ?). 
Source: Health Quality Ontario Health Technology Assessment.’”” 


Costs of Usual Care and Follow-Up 


As in our prior analysis,’” we accounted for the cost of medication and follow-up used in usual 
care and in certain patients (e.g., 48-50% of people with moderately severe disease take 
medications). We based the costs of follow-up by physicians on the fee-for-service schedule 
and number of visits. As per the 2016 CANMAT guidelines”? and expert consultation,’’ the 
number of follow-up visits depended on the clinical pathway: 


e People on medication were assumed to have biweekly follow-up visits with a general 
practitioner in the first month and then monthly visits until the end of the time horizon. 
We assumed that the consultation with a psychiatrist to update the initial medication plan 
occurred once, at the completion of face-to-face CBT (at 4 months) 

e People who received no medication were assumed to have follow-up visits with a 
general practitioner every 6 months 


We based drug costs on the Ontario Drug Benefit program after accounting for dispensing fees 
and markup costs, as suggested in the literature.’” 


Tables 15 and 16 describe input values for cost parameters used in calculations of total yearly 
costs per person, related to medication use and follow-up by a GP in the usual care or other 
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intervention strategies in the reference case analysis. It was assumed that drug coverage is 
publicly funded in 57% of cases based on the 2017 Canadian report by Sutherland and Thy.'” 
The costing of medications was also adjusted for patient adherence. It was assumed that 50% 
of people with mild to moderate major depression or anxiety take medications at baseline”? or 
when prescribed for the first time during the follow-up.We assumed that the compliance is 100% 
in people who were on medication at baseline and had to switch to a higher dose in the second 
half of the year. This usual care medication pathway was accounted for in individuals on CBT 
treatment in a scenario analysis. In the reference case analysis, we assumed that people would 
start with medication after no response to iCBT or CBT treatment (all pathways shown in 

Table 16). 


Table 15: Costs Used in the Economic Model: Usual Care and Follow-Up 


Model Parameter Mean, $? _ Distribution Source 
One-time costs related to CBT 

Initial assessment, total 223.35 Fixed HQO HTA’”’ 

GP referral 62.75 — OHIP code K005, 30 min''° 
Assessment/consultation by psychiatrist 80.30 — OHIP code K197, 30 min''° 


or psychologist 


Weekly costs of pharmacotherapy” 


Sertraline, 50 mg/d 6.85 Fixed ODB"""; Kolber (2014);1*? HQO 
77 
Sertraline, 100 mg/d 7.44 = HTA 


Abbreviations: CBT, cognitive behavioural therapy; GP, general practitioner; HQO, Health Quality Ontario; HTA, health technology assessment; 
ODB, Ontario Drug Benefit Program; OHIP, Ontario Health Insurance Plan Schedule of Benefits and Fees. 

All costs were estimated in 2018 Canadian dollars. 

’Costs of drugs include a dispensing fee of $10.22 and a mark-up of $1.71 for 30-day use.'!2 


Table 16 describes the annual costs of additional care, including medications, for the four 
different pathways considered in the model. The reference case analysis considered costs 
related to two clinical pathways for additional or usual care: costs associated with people who 
were not on medication and costs of adding medication during follow-up. A separate model and 
scenario considered the costs of additional care, including all four possible clinical pathways. 
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Table 16: Estimates of Annual Per-Person Costs of Additional Care Including Medications? 


Additional Care in Per-Person Cost, $* 


UC/CBT Physician Drug 1° Drug 2° Total¢ 
Not using drugs 125.5 NA NA 125.5 
Same medication 564.75 356.2 NA 666.3 
throughout year 

Starting with one 878.5 178.1 193.44 1,039.5 


medication and adding 

another during follow- 

up 

Adding medication 564.75 178.1 NA 615.5 
during follow-up 


Abbreviations: CBT, cognitive behavioural therapy; NA, not available; UC, usual care. 

4All costs were estimated in 2018 Canadian dollars. 

PAIl costs related to physician fees and medication are considered fixed in the analysis, meaning that no distributions were assigned. 

°Total costs are not a simple sum of the values across the rows; they are smaller because they are adjusted to account for public funding of 
drugs in 57% of the population, for compliance, and the increasing number of follow-up physician visits after changes in medication dose 
during the last 6 months of follow-up. Another medication dose was considered during follow-up for people who did not benefit from 
treatment. 


Societal Perspective 


In a scenario analysis, we used a societal perspective and included the following costs: 


e Out-of-pocket costs 
e Cost of people’s time spent travelling and receiving CBT 
e Cost of lost productivity 


For out-of-pocket costs, we considered the cost of parking at the hospital. We estimated the 
number of days patients receive CBT in person and applied a maximum daily parking rate of 
$26 ($390 for 15 days of therapy). The public transport costs were assumed to range from 
$4 and $6 (median $5 used in the estimation of the travel costs: $75). 


We assumed that people spend about 2 hours traveling to and from the hospital and an 
additional 2 hours to receive CBT treatment. To value the time that people spend in face-to-face 
CBT, we used the Ontario minimum wage of $11.60 per hour plus a 30% employee benefit 
(about $15.10 per hour). We assumed that people would be able to work part time while 
receiving face-to-face CBT; thus, the value of time spent on face-to-face CBT totaled about 

$60 per day of treatment ($904 for 15 days of therapy). 


We assumed that people would not be able to work full time while receiving face-to-face CBT 
and used the median income in Ontario to estimate the cost of lost productivity. Assuming an 
income of $33,840 (plus 30% benefits), we calculated a $22.67 hourly rate. The cost of 
productivity loss associated during CBT treatment was about $1,275 (assuming 15 days of 
therapy and part time work including 56.25 hours spent away from work: 56.25 x $22.67 = 
$1,275). 
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Table 17: Per-Person Societal Costs Associated With Receiving Face-to-Face Cognitive 
Behavioural Therapy 


Model Parameter Mean? Distribution® Source 


Productivity loss, per session 


Hourly rate based on median income in $22.67 NA Statistics Canada, 201619 
Ontario of $34,000/yr with 30% benefits 
($1,950 hr/yr) 


Costs associated with productivity loss $1,275 — — 


Daily travel and wage costs 


Parking $26 NA 

Estimated 
Public transport $5 NA 
Traveling and treatment time 4hr NA Assumption 
Minimum wage in Ontario $15/hr NA Ministry of Labor, 2017''4 
Total travel and wage costs (15 days) 
Parking costs (conservative estimate) $390 NA NA 
Public transport $75 NA 
Wage costs $904 NA 
Total societal costs (CBT including travel $3,454 Gamma’ NA 
costs, conservative estimate) 
Total societal costs (CBT including public $2,254 Gamma’ NA 
transportation travel costs) 
Total societal costs (CBT only) $2,179 Gamma* NA 


Abbreviations: CBT, cognitive behavioural therapy; NA not applicable. 
*Original nondiscounted costs in 2018 Canadian dollars. 
bUsed in probabilistic sensitivity scenario analyses; standard error (SE) based on an assumption that mean costs vary by +25%. 


°Gamma distributions were assigned in probabilistic sensitivity analysis, assuming SE to the mean cost of 25%. Two parameters of the gamma 
distribution (a, A) are derived from the mean and SE. Formulas for these calculations are: a = (Mean?)/(SE?); A= Mean/([Mean x SE] 2). 


Analysis 


The primary outcome of the economic evaluation is the ICER, reported as the incremental cost 
per QALY gained. Following CADTH guidelines,®* we used probabilistic sensitivity analysis to 
address both primary and secondary economic research questions. The reference case 
analysis simulated 100,000 trials. Uncertainty in the model estimates was also evaluated using 
a series of probabilistically computed scenario analyses for some important parameters (e.g., 
utilities associated with internet and face-to-face CBT, efficacy of iCBT, dropout rates, iCBT 
costs). The analyses are conducted using TreeAge Pro 2018.R1 (TreeAge Software, 
Williamstown, MA). 


Generalizability 


The findings of this economic analysis may not be generalizable to all patients with major 
depression or anxiety disorder. Internet-delivered CBT may not be suitable for people who are 
suffering from severe mood or anxiety disorders that are unlikely to respond to outpatient 
treatment. These may require specialized multidisciplinary intensive outpatient or inpatient care. 
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Expert Consultation 


Expert consultations were conducted throughout the review process to feed into the 
development and refinement of the research questions, review methods, and review results, as 
well as to contextualize the evidence on iCBT to Ontario. Expertise was sought from clinical and 
methodological experts within the CBT field. 


Results 


Below we present the results of our primary economic evaluation: the base case and sensitivity 
analyses. 


Reference Case Analysis 


We present results of two cost-utility analyses, for adults with mild to moderate major 
depression (Table 18), and adults with anxiety (Table 19). 


Cost-Utility Analysis for Mild to Moderate Major Depression 


In adults with mild to moderate major depression, in a sequential cost—utility analysis ranking 
five strategies by increasing costs, unguided iCBT and group CBT were dominated and 
excluded from final calculations. Of note, QALYs associated with all guided iCBT and in-person 
CBT were similar. Compared with usual care, guided iCBT (i.e., guided via a web portal with a 
therapist who is a non-physician, regulated health care professional) was associated with an 
increased quality-adjusted survival of 0.04 QALYs (95% credible interval [Crl] 0.01-0.07) and 
increased mean costs of $1,257 (95% Crl $630—-$2,061), yielding an ICER of $31,575 per QALY 
gained. 


Table 18: Cost—Utility Analysis, Sequential Approach: Internet-Delivered CBT, In-Person CBT, and 
Usual Care for Major Depression 


ICER, 

Mean Incremental Incremental $/QALY 
Strategy Mean Costs, $2 QALYs Costs, $2 QALYs° Gained? 
Unguided iCBT 274.65 0.778 — — — 
Usual care 409.40 0.787 134.75 0.009 14,972 
Guided iCBT 1,666.26 0.826 1,256.86 0.039 31,575 
(therapist) 
Group CBT 1,691.23 0.817 24.97 —0.009 Dominated? 
Individual CBT 2,654.00 0.826 987.79 0.00001 59,377,114° 


Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; ICER, incremental cost-effectiveness ratio; QALY, quality- 
adjusted life-year. 

All costs in 2018 Canadian dollars. No discounting was done given the 12-month time horizon. 

Incremental cost = average cost (strategy of guided CBT) — average cost (strategy of usual care). 

‘Incremental effect = average effect (strategy of guided CBT) - average effect (strategy of usual care). 

4Negative ICER means this strategy was considered dominated and was excluded from the analysis. 

°This estimate of the high ICER implies that individual CBT was just slightly better when it comes to QALY increments and is essentially dominated 
by guided iCBT because it provides benefit almost equal to that of guided iCBT, but it is more costly. 

Note: Results may appear incorrect because of rounding. 


In the reference case analysis including adults with major depression, the probability of cost- 
effectiveness of guided iCBT was 63% at a willingness-to-pay value of $50,000/QALY and 
increased to 67% at a willingness-to-pay value of $100,000/QALY (Figure 4). The probability of 
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cost-effectiveness of guided iCBT was the highest (87% at willingness-to-pay of $50,000/QALY) 
when we assumed that guided iCBT was provided solely by the employer. 
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Figure 4: Cost-Effectiveness Acceptability Curve: Internet-Delivered CBT, In-Person CBT, 
and Usual Care for Mild to Moderate Major Depression 


Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; QALY, quality-adjusted life-year; WTP, willingness to pay. 

Note: The cost-effectiveness acceptability curve graphically presents the probability of cost-effectiveness of the examined CBT strategies and usual 
care across various willingness-to-pay values. The x-axis shows the probability of cost-effectiveness (range: 0 to 1) and the y-axis represents 
willingness-to-pay values (range: $0 to $100,000 per QALY gained). 


All other analyses that considered guided iCBT by a different type of therapist (i.e., coach- 
guided) or a different type of provider (i.e., employer-led) identified guided iCBT as the best 
option of all strategies with the following ICERs (compared with usual care): 


e $17,092 per QALY if guided iCBT was provided solely by the employer 

e $29,651 per QALY if guided iCBT was provided by a coach-therapist 

e $31,691 per QALY if a higher cost range was used for all iCBT strategies (cost 
estimates presented in Table 13) 


In addition, the results remained robust in the sensitivity analysis considering the lower costs of 
delivery of individual or group CBT, showing guided iCBT as the optimal strategy, with an ICER 
for guided versus unguided iCBT of $39,670 per QALY (Appendix 7, Table A11). 


Finally, the results remained robust in the sensitivity analysis considering that a portion of 
individuals take medications from baseline (Table 19). Guided iCBT was the optimal strategy, 
with an ICER for guided iCBT versus usual care of $33,397 per QALY (Appendix 8, Table A13). 


Cost-Utility Analysis for Anxiety Disorders 


In the second population of adults with an anxiety disorder, a sequential cost-utility analysis that 
ranked five strategies by increasing costs, usual care was dominated by unguided iCBT and 
group CBT was excluded by extended dominance from the final calculations (ICER of group 
CBT vs unguided iCBT: $65,000 per QALY). If compared with usual care (i.e., the only publicly 
funded option), guided iCBT was associated with an ICER of $26,719/QALY gained. Compared 
with unguided iCBT, guided iCBT (i.e., therapist-guided via web portal) was associated with an 
increased quality-adjusted survival of 0.03 QALYs (95% Crl 0.01—0.06) and increased mean 
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costs of $1,395 (95% Crl $767-$2,192), yielding an ICER of $43,214 per QALY gained 
(Table 19). 


Table 19: Cost—Utility Analysis, Sequential Approach: Internet-Delivered CBT, In-Person CBT, and 
Usual Care for Anxiety Disorders 


Mean Incremental Incremental ICER, $/QALY 
Strategy Mean Costs, $3 QALYs Costs, $2° QALYs° Gained? 
Unguided iCBT 269.70 0.802 — — — 
Usual care 409.40 0.787 139.71 -0.015 Dominated? 
Group CBT 1,275.37 0.817 —_ _— Extended 
dominance® 
Guided iCBT 1,665.17 0.834 1,395.47 0.032 43,214 
Individual CBT 1,952.78 0.8267 287.61 -0.007 Dominated? 


Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; ICER, incremental cost-effectiveness ratio; QALY, quality- 


adjusted life-year. 
All costs in 2018 Canadian dollars. No discounting was done because of the 12-month time horizon. 


>Incremental cost = average cost (strategy of guided CBT) — average cost (strategy of unguided iCBT). 
‘Incremental effect = average effect (strategy of guided CBT) — average effect (strategy of unguided iCBT). 
4The negative ICER, this and usual care strategies were considered dominated. 

°ICER of group CBT vs. usual care: $65,070/QALY. 

Note: Results may appear incorrect because of rounding. 


In the reference case analysis including adults with anxiety disorders, the probability of cost- 
effectiveness of guided iCBT was 49% at willingness-to-pay of $50,000 per QALY gained and 
increased to 70% at willingness-to-pay of $100,000 per QALY gained (Figure 5). The probability 
of cost-effectiveness of guided iCBT was the highest (80% at a willingness-to-pay value of 
$50,000/QALY) when we assumed that guided iCBT was solely provided by the employer. 


1 
fe) 
5 08 
@ 
+ 0.6 
7 
+ 0.4 
fe} 
O 0.2 
x 
0 
Q LD LH VO LO OC LO BO BOC LC LC LC PLC PG BVO GO LQ VG VQ LQ L 
OT On Oa Ors Or es Oe. © nT Os Oe Os 0 OS OOS OO eS OO SS) 
YY O SM XD A DP ON LY DY LY O SS OF OH 
SPP PPP PE OPS COO SP PP 
WTP($/QALY) 
=== Group CBT e=——_l|ndividual CBT === Therapist-guided iCBT 
ee Unguided iCBT e=== Usual care 


Figure 5: Cost-Effectiveness Acceptability Curve: Internet-Delivered CBT, In-Person CBT, 

and Usual Care for Anxiety Disorders 
Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-guided CBT; QALY, quality-adjusted life-year; WTP, willingness to pay. 
Note: The cost-effectiveness acceptability curve graphically presents the probability of cost-effectiveness of the examined CBT strategies and usual 
care across various willingness-to-pay values on the x—y coordinate system. The x-axis shows the probability of cost-effectiveness (range: 0-1) and 
the y-axis represents willingness-to-pay values (range: $0-$100,000 per QALY gained). 
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All other analyses that considered guided iCBT by a different type of therapist (i.e., coach- 
guided) or a different type of provider (i.e., employer-led) identified guided iCBT as the best 
option of all strategies with an ICER (comparing it with unguided iCBT) of: 


e $25,349 per QALY if guided iCBT was provided solely by the employer 
e $40,960 per QALY if guided iCBT was provided by a coach-therapist 
e $40,069 per QALY if a higher cost range was used for all iCBT strategies (Table 13) 


In addition, in the sensitivity analysis considering lower costs of delivery of individual or group 
CBT, group CBT and guided iCBT were economically attractive with the ICERs below $50,000 
per QALY. The ICER of group CBT versus usual care was $38,515 per QALY (incremental 
mean costs of $597 and incremental mean QALYs of 0.015) and the ICER for guided iCBT 
versus group CBT was $47,753 (incremental mean costs of $800 and incremental mean QALYs 
of 0.017) (Appendix 7, Table A12). 


The results remained robust in the sensitivity analysis considering that a portion of individuals 
take medications from baseline (see Table 16). Guided iCBT was the optimal strategy, with the 
ICER of guided versus unguided iCBT of $43,250 per QALY (Appendix 8, Table A14). 


Sensitivity Analysis: Reference Case Analysis 
Coach-Based Ontario Program (Scenario A) 


The coach-based program used as an example for this scenario assumed a probability of 
response to iCBT at 40%. We found that all guided CBT programs (therapist or face to face) 
were associated with increments in QALYs and costs as compared to each other (in a ranking 
order). However, the estimated ICERs were around or below a willingness-to-pay value of 
$100,000 per QALY gained (Table 20). The probability of this program being cost-effective 
compared to other options was 48% at a willingness-to-pay value of $50,000 per QALY gained 
and 28% at a willingness-to-pay value of $100,000 per QALY gained. 


Table 20: Cost—Utility Analysis, Sequential Approach: Internet-Delivered CBT, In-Person CBT, and 
Usual Care for Major Depression 


Mean Incremental Incremental ICER, $/QALY 

Strategy Mean Costs, $2 QALYs Costs, $2 QALYs° Gained? 
Unguided iCBT 381.62 0.777 

Usual care 409.40 0.787 27.78 0.0092 3,004 
Ontario coach-based 989.321 579.92 0.0197 29,387 
iCBT 0.806 

Group CBT 1,690.15 0.817 700.83 0.0105 66,836 
Individual CBT 2,654.08 0.827 963.93 0.0096 100,312 


Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; ICER, incremental cost-effectiveness ratio; QALY, quality-adjusted 
life-year. 

All costs in 2018 Canadian dollars. No discounting was done because of the 12-month time horizon. 

Incremental cost = average cost (strategy of individual CBT) — average cost (strategy of group CBT). 

‘Incremental effect = average effect (strategy of individual CBT) — average effect (strategy of group CBT). 

Note: Results may appear incorrect because of rounding. 
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Physician-Guided iCBT (Scenario B) and Company-Led Program (Scenario C) 


Both scenarios found that guided iCBT was the optimal strategy. In scenario B, the ICER was 
$34,625 per QALY gained (guided iCBT versus usual care) in adults with major depression and 
$47,045 per QALY gained (guided vs unguided iCBT). In scenario C, the ICER was $16,790 per 
QALY (guided iCBT vs usual care) in adults with major depression, and $33,435 per QALY 
gained in adults with anxiety disorders (guided vs unguided iCBT). 


Additional Scenario: Societal Perspective 


Guided iCBT was the optimal strategy for adults with major depression, associated with an 
ICER of $31,660 per QALY gained (Table 21). The probability of guided iCBT being cost- 
effective as compared to other strategies was 78% and 92% at willingness-to-pay values of 
$50,000 and $100,000 per QALY gained. Similar findings were found in a second analysis that 
included adults with anxiety disorders. 


Table 21: Cost—Utility Analysis, Sequential Approach: Internet-Delivered CBT, In-Person CBT, 
and Usual Care for Major Depression, From the Societal Perspective 


Mean Costs, Mean Incremental Incremental ICER, $/QALY 
Strategy gab QALYs Costs, $°° QALYs? Gained? 
Unguided iCBT 274.53 0.777 _— — — 
Usual care 409.40 0.787 134.79 0.009 14,551 
Guided iCBT 1,667.41 0.826 1,258.80 0.039 31,660 
Group CBT 3,574.92 0.817 — — Dominated 
Individual CBT 4,179.31 0.826 2,511.80 0.000 Dominated® 


Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; ICER, incremental cost-effectiveness ratio; QALY, quality- 
adjusted life-year. 

*Costs estimated in Table 16, additional societal costs associated with in-person CBT of $2,179. 

PAIl costs in 2018 Canadian dollars. No discounting was done because of the 12-month time horizon. 

‘Incremental cost = average cost (strategy of guided CBT) — average cost (strategy of usual care). 

4Incremental effect = average effect (strategy of guided CBT) — average effect (strategy of usual care). 

This is essentially dominated by guided iCBT because, after rounding, it provides a benefit equal to that of guided iCBT but is more costly; the 
estimated ICER for individual CBT vs. guided iCBT would be approximately $65 million if we used an actual increment in QALYs of 0.0004. 
Note: Results may appear incorrect because of rounding. 


Additional Sensitivity Analyses 


We performed additional sensitivity analyses for each parameter and found that, while the cost- 
effectiveness results remained robust to changes in dropout rates from iCBT or CBT strategies, 
they were sensitive to changes in the probability of recovery and changes in the utility weight 
associated with guided iCBT. 


Influential Parameters 


Recent findings of the UK IAPT program’**6"' suggested that the recovery rate of guided iCBT 
could be much higher than the rate found in the published evidence. Thus, if the recovery rate 
with guided iCBT increased to 60% from the 48% found in the reference case for major 
depression, the ICER would substantially decrease, from $31,575 to $25,100 per QALY gained 
(Table 22). In addition, the probability of cost-effectiveness of guided iCBT over other therapies 
would increase to over 90% at a willingness-to-pay value of $50,000 per QALY gained (from 
63% in the reference case analysis). 
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Table 22: Sensitivity Analysis, Sequential Approach: Increased Recovery Rate With Guided iCBT 
Mean Costs, Mean Incremental Incremental ICER, $/QALY 

Strategy gab QALYs Costs, $°° QALYs? Gained> 

Unguided iCBT 274.62 0.777 — — — 
Usual care 409.40 0.787 134.78 0.009 14,644 
Guided iCBT 1,660.09 0.838 1,278.64 0.051 25,110 

Group CBT 1,688.05 0.817 — — Dominated 
Individual CBT 2,657.27 0.827 _ _— Dominated 


Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; ICER, incremental cost-effectiveness ratio; QALY, quality- 


adjusted life-year. 

*Costs estimated in Table 16, additional societal costs associated with in-person CBT of $1,244. 

PAIl costs in 2018 Canadian dollars. No discounting was done because of the 12-month time horizon. 
‘Incremental cost = average cost (strategy of guided CBT) — average cost (strategy of usual care). 
dincremental effect = average effect (strategy of guided CBT) — average effect (strategy of usual care). 
Note: Results may appear incorrect because of rounding. 


Publication bias has been seen in some studies.” If the utility weight of guided iCBT therapy 
decreased by 5% (from 0.83 in the reference case to 0.79), the ICER of guided iCBT versus 
usual care would increase to about $39,200 per QALY gained (compared to $31,575 in the 
reference case). The ICER of group versus guided CBT would be smaller, resulting in exclusion 
of guided iCBT from the calculations by extended dominance. Group CBT versus usual care is 
associated with an ICER of about $43,000 per QALY gained and becomes less favorable as 
compared to individual CBT (with the ICER of about $38,700 per QALY gained). Individual CBT 


versus usual care becomes optimal associated with an ICER of about $56,000 per QALY 


gained when compared with usual care (Table 23). 


Table 23: Sensitivity Analysis, Sequential Approach: Change in Utility Weight Associated With 


Guided iCBT 
ICER, cont. 
Mean Incremental Incremental ICER, $/QALY $/QALY 
Strategy Mean Costs, $? QALYs Costs, $2 QALYs° Gained? Gained?4 
Unguided iCBT 274.62 0.777 _— —_— —_— —_— 
Usual care 409.40 0.787 134.79 0.009 38,931 
Guided iCBT 1,666.86 0.81701 1,257.46 0.030 39,184 Extended 
dominance 
Group CBT 1,690.15 0.81704 23.29 0.00003 39,162¢ 42,700 
Individual CBT 2,654.08 0.8266 963.93 0.0096 38,678 56,115 


Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; ICER, incremental cost-effectiveness ratio; QALY, quality-adjusted 
All costs in 2018 Canadian dollars. No discounting was done because of the 12-month time horizon. 

Incremental cost = average cost (strategy of guided CBT) — average cost (strategy of usual care). 

‘Incremental effect = average effect (strategy of guided CBT) — average effect (strategy of usual care). 

‘The ICER of group CBT vs. usual care changed to $42,700/QALY after exclusion of guided iCBT by extended dominance. 

Note: Results may appear incorrect because of rounding. 


life-year. 


Additionally, the probability of cost-effectiveness of guided iCBT over other therapies would 
decrease to 38% at a willingness-to-pay value of $50,000 per QALY gained (from 63% in the 
reference case; Figure 6). In this analysis, the probability of cost-effectiveness of all CBT 
therapies, when compared to each other, remains quite low (up to 40%) over a wide range of 
willingness-to-pay values. 
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Figure 6: Cost-Effectiveness Acceptability Curve: Internet-Delivered CBT, In-Person CBT, and 
Usual Care for Major Depression, a Sensitivity Analysis Assuming a 5% Decrease in the 
Utility Weight Associated With Guided iCBT 


Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; QALY, quality-adjusted life-year; WTP, willingness to pay. 

The cost-effectiveness acceptability curve graphically presents the probability of cost-effectiveness of the examined CBT strategies and usual care 
across various willingness-to-pay values on the x-y coordinate system. The x-axis shows the probability of cost-effectiveness (range: 0-1) and the y- 
axis represents willingness-to-pay values (range: $0 to $100,000 per QALY gained). 


Secondary Research Questions — Exploratory Scenario Analyses 


Lifetime Cost-Effectiveness of Guided iCBT Within a Stepped-Care Model for 
Adults With Mild to Moderate Major Depression as Primary Diagnosis 


Both strategies that included guided iCBT were cost saving and associated with increments in 
QALYs when compared with usual care. Over a lifetime time horizon using a stepped-care 
model, guided iCBT followed by individual face-to-face CBT represented an economically 
attractive option when compared with guided iCBT followed by group CBT (ICER of 
$1,100/QALY gained). The results remained robust in scenario analyses addressing parameter 
uncertainty (e.g., changes in the efficacy of iCBT and CBT, dropout rates, utilities, and 
medication costs) and duration of time horizon. We present detailed results of this analysis in 
Appendix 9. 


Unguided Followed by Guided iCBT 
Adults With Mild to Moderate Major Depression 


In adults with mild to moderate major depression, a sequential cost—utility analysis ranking four 
strategies by increasing costs found that all CBT strategies were associated with increments in 
cost and benefits as compared with usual care (Table 24). Compared with usual care, unguided 
iCBT followed by guided CBT was associated with an increased quality-adjusted survival of 0.02 
QALYs and increased mean costs of $306, yielding the most favourable ICER, $19,454 per 
QALY gained. Further, the ICER for group CBT versus iCBT was about $67,000 per QALY 
gained, and the ICER for individual versus group CBT was about $100,000 per QALY gained. 
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Table 24: Cost-Utility Analysis: Stepped-Care iCBT, In-Person CBT, and Usual Care for Major 


Depression 
ICER: Strategy 

Mean Incremental Incremental vs. Usual Care, 
Strategy Mean Costs, $3 QALYs Costs, $?° QALYs° $/QALY Gained 
Usual care 409.40 0.7869 -- -- -- 
Unguided followed 715.41 0.8026 306.00 0.0157 19,454 
by guided iCBT 
Group CBT 1,690.18 0.8171 974.77 0.0145 67,161 
Individual CBT 2,654.39 0.8267 964.22 0.0096 100,316 
Abbreviations: CBT, cognitive behavioural therapy; Crl, credible interval; ICER, incremental cost-effectiveness ratio; QALY, quality-adjusted life- 


year. 
4All costs in 2018 Canadian dollars. No discounting was done because of the 12-month time horizon. 
Incremental cost = average cost (strategy unguided CBT) — average cost (strategy usual care). 
‘Incremental effect = average effect (strategy guided CBT) — average effect (strategy usual care). 
Note: Results may appear incorrect because of rounding. 


In analyses that included adults with major depression, the probability of cost-effectiveness of 
unguided followed by guided iCBT in those who tend to dropout of unguided iCBT was 48% ata 
willingness-to-pay value of $50,000 per QALY gained. This decreased to 27% at a willingness- 
to-pay value of $100,000 per QALY gained (Figure 7). At these two willingness-to-pay values, 
the probability of cost-effectiveness of group CBT was 28% and 33%, and individual CBT was 
10% and 39%. If the dropout rate from unguided iCBT raised from 32% (in the base case) to 
75%, the probability of cost-effectiveness of unguided followed by guided iCBT was 94% ata 
willingness-to-pay of $50,000/QALY and of $100,000/QALY. 
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Figure 7: Cost-Effectiveness Acceptability Curve: Internet-Delivered CBT (Unguided Followed by 
Guided), In-Person CBT, and Usual Care for Mild to Moderate Major Depression 


Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; QALY, quality-adjusted life-year; WTP, willingness to pay. 

The cost-effectiveness acceptability curve graphically presents the probability of cost-effectiveness of the examined CBT strategies and usual care 
across various willingness to pay values on the x-y coordinate system. The x-axis shows the probability of cost-effectiveness (range: 0-1) and the y- 
axis represents willingness to pay values (range: $0 to $100,000 per QALY gained). 
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Adults With Anxiety Disorders 


In adults with an anxiety disorder, a sequential cost-utility analysis ranking four strategies by 
increasing costs found that group CBT was dominated by stepped-care iCBT (Table 25). 
Compared with usual care, unguided iCBT followed by guided CBT was associated with an 
increased quality-adjusted survival of 0.04 QALYs and increased mean costs of $301, yielding 
the most favorable ICER of $7,670 per QALY gained. Further, the ICER of individual CBT 
versus stepped-care iCBT was over $2 million per QALY gained due to a small increment in 
QALYs with individual CBT. Stepped-care iCBT, therefore, seems to be the optimal economic 
option for the initial treatment of adults with anxiety. 


Table 25: Cost-Utility Analysis: Stepped-Care iCBT, In-Person CBT, and Usual Care for Anxiety 


Mean Incremental Incremental ICER 
Strategy Mean Costs, $2 QALYs Costs, $2" QALYs° $/QALY Gained 
Usual care 409.40 0.787 — — — 
eee eee 710.62 0.826 301.21 0.039 7,667 
Group CBT 1,276.23 0.817 565.62 —-0.009 Dominated 
Individual CBT 1,954.99 0.827 1,244.37 0.001 2,223,063 


Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; ICER, incremental cost-effectiveness ratio; QALY, quality- 


adjusted life-year. 
All costs in 2018 Canadian dollars. No discounting was done because of the 12-month time horizon. 


Incremental cost = average cost (strategy of guided CBT) - average cost (strategy of usual care). 
‘Incremental effect = average effect (strategy of guided CBT) - average effect (strategy of usual care). The exact incremental QALYs were about 


0.00056, which we rounded to 0.001. 
Note: Results may appear incorrect because of rounding. 


In an analysis including adults with anxiety disorders, the probability of cost-effectiveness of 
unguided followed by guided iCBT in those who tend to dropout from unguided iCBT was 86% 
at a willingness-to-pay value of $50,000 per QALY gained. This decreased to 75% ata 
willingness-to-pay value of $100,000 per QALY gained (Figure 8). At these two willingness-to- 
pay values, the probability of cost-effectiveness of group CBT was 8% and 7% and individual 
CBT was 5% and 18%, respectively. If the dropout from unguided iCBT raised from 32% (in the 
base case) to 75%, the probability of cost-effectiveness of unguided followed by guided iCBT 
was 70% and 72% at willingness-to-pay values of $50,000 and $100,000 per QALY gained, 


respectively. 
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Figure 8: Cost-Effectiveness—Acceptability Curve: Internet-Delivered CBT, In-Person CBT, and 
Usual Care for Anxiety Disorders 

Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; QALY, quality-adjusted life-year; WTP, willingness to pay. 

The cost-effectiveness acceptability curve graphically presents the probability of cost-effectiveness of the examined CBT strategies and usual care 


across various willingness-to-pay values on the x—y coordinate system. The x-axis shows the probability of cost-effectiveness (range: 0-1) and the y- 
axis represents willingness-to-pay values (range: $0 to $100,000 per QALY gained). 


Discussion 


Our analysis employed several decision-modeling approaches to examine the cost- 
effectiveness of iCBT for the treatment of mild to moderate major depression or anxiety 
disorders. We showed that guided iCBT may represent an economically attractive strategy 
option when used as a one-time, non-repetitive short-term treatment or when combined with in- 
person CBT over a long-term time horizon. In all analyses, guided iCBT was associated with 
ICERs of less than $50,000 per QALY gained. 


When all CBT strategies and usual care were compared using a sequential approach in the 
reference case cost-—utility analyses, guided iCBT was selected as the best ranking option, while 
its clinical benefits (QALYs) and those associated with in-person CBT strategies were similar. 
For instance, over a 1-year time horizon in adults with mild to moderate major depression, 
guided iCBT was associated with increases in mean QALY of 0.04 (95% credible interval [Crl] 
0.01—0.07) and mean costs of $1,257 (95% Crl $630 to $2,061) compared with usual care (the 
ICER: about 31,000 per QALY gained). Similar results were shown in the cost-utility analysis in 
adults with anxiety disorders (the ICER for guided vs unguided iCBT was about $43,000 per 
QALY gained). Over a short-term time horizon, uncertainty around the cost-effectiveness 
estimates was moderately high, depending on the maximum amount a decision-maker would be 
willing to pay for this treatment (e.g., about 60% for major depression and about 50% for anxiety 
at $50,000 per QALY gained). 


We conducted several sensitivity analyses to explore the robustness of the reference case 
results. In most analyses, the ICER for guided iCBT remained robust and at similar probabilities 
of cost-effectiveness below the commonly used willingness to pay value of $50,000 per QALY 
gained. However, the probability of cost-effectiveness substantially increased to over 80% ata 
willingness to pay value of $50,000 per QALY gained if the cost of guided iCBT decreased by 
about half (a scenario analysis with employer-led guided iCBT). In addition to the cost of 
delivery, a decrease in the probability of response to guided iCBT versus usual care from 
around 70% to 40% as reported in scenario A, would lead to a very high uncertainty around the 
estimate of ICER (i.e., 40% chance of guided iCBT to be cost-effective at willingness to pay of 
$50,000/QALY and 28% of guided iCBT to be cost-effective at willingness to pay of 
$100,000/QALY). For the reference case analysis, the recovery rate after guided iCBT was 
assumed to be around 48% based on estimates reported in a study”! included in our review.?' 
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Recovery rate is an important outcome and real-word data from the UK Improving Access for 
Psychological Therapies (IAPT) program’?®6''5 suggest that guided iCBT could be associated 
with much higher recovery rates. Our sensitivity analysis addressing this change, assuming an 
increase in the recovery rate of 60% (as shown in the UK IAPT program’?®°""5), suggests a 
substantially large increment in certainty around the ICER estimate of approximately 90% at a 
willingness to pay of $50,000/QALY. If the improvement in health state utility after the treatment 
with iCBT was smaller by 5%, then guided iCBT would not represent the best ranking option 
and would be dominated by group CBT (see Table 23). This finding indicates the importance of 
determining a true change in the utility weight for health states when guided iCBT is used 
instead of in-person CBT, as the evidence is very limited, and the studies included in our review 
suggested very small differences in patients’ quality of life between iCBT and in-person CBT 
options. Further studies need to examine the advantages of iCBT with respect to overcoming 
barriers to access and stigma, as well as its convenience of use and the correlation of these 
specific iCBT features with changes in quality of life for different subgroups of individuals 
suffering from major depression and anxiety disorders. 


Although our secondary analyses were more complex and exploratory in nature, their findings 
are in line with the results of the reference case analysis. They show that guided iCBT is optimal 
and economically attractive over a patient’s lifetime when delivered as the first-step of a 
stepped-care model and is followed by in-person CBT for people who have recurrent episodes 
of anxiety or major depression (the ICER, depending on the format of in-person CBT and the 
comparator, was either cost saving or $1,100 per QALY gained). Another secondary analysis 
exploring a subsequent delivery of guided iCBT in those who tend to drop out from unguided 
iCBT also showed that a stepped-care iCBT approach represents good value for money versus 
usual care (ICER: $19,500 per QALY gained). This analysis partially addressed the issue 
related to high attrition rates in people on unguided iCBT, including a lack of understanding of 
patient readiness for iCBT. 


Equity 


Barriers to in-person CBT include stigma, geography, time, cost, and lack of treatment fidelity. 
Access to specialized providers varies depending upon where someone lives within Ontario. 
Internet-delivered CBT represents an additional option that may help overcome these barriers 
and potentially increase access to treatment.''® 


Although iCBT reduces barriers to access to mental health services, it is important to 
acknowledge that people who are not comfortable with technology may not be ideal candidates 
for iCBT. However, recent patient-preference surveys® suggest that iCBT guided by therapists 
could become a preferred option over in-person CBT or medication. Current therapist-guided 
iCBT is developed in English language and is available in French as of July 3, 2018 

(P. Farvolden, PhD, written and oral communications, January to July 2018); however, some 
inequities may remain for recent immigrants and non-native English or French speakers. ''7""8 
Tailoring iCBT programs for different populations, including Aboriginal and First Nation 
populations, will be required to enable equal and effective access to all eligible people with 
depression and anxiety. 
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Limitations 


Although we used a comprehensive analytic approach to this economic analysis, we recognize 
several limitations: 


e Our analysis did not examine the effectiveness and cost-effectiveness of any Ontario- 
specific iCBT program; our input parameters related to the effectiveness of iCBT were 
based on the literature. The clinical utility of guided iCBT should be carefully examined 
during the procurement and implementation process (with weekly monitoring of clinical 
outcomes as implemented in the [APT program), given the wide variety of commercial 
and non-commercial iCBT packages available 

e Inthe reference case, the effectiveness of iCBT, based on the published evidence, was 
lower than estimates recognized in real-word effectiveness programs such as the UK 
Improving Access for Psychological Therapies program (IAPT).72°°''5 However, the 
cost-effectiveness of guided iCBT based on finding of the UK IAPT program (considered 
in a scenario analysis) was in line with the results of reference case analysis 

e The evidence is limited for the relative effectiveness of guided iCBT versus in-person 
CBT because there were no large noninferiority studies published in the literature. We 
made potentially strong assumptions related to the probability of response after guided 
iCBT, assuming no difference between guided iCBT and face-to-face CBT. However, our 
estimates related to the probability of recovery reflected differences in the effects 
between these two CBT treatments. To account for the limitations of the evidence, we 
defined the use of iCBT in our modeling studies as one-time, non-repetitive therapy 
provided within a short time frame. Future studies need to confirm the equivalence of 
face-to-face CBT and guided iCBT for the range of target populations that we examined 
in our analyses 

e The costs associated with delivery of iCBT are based on plausible but hypothetical 
scenarios potentially overestimating the costs of training and supervision for the 
reference case analysis considering therapist-guided iCBT. Nevertheless, we made 
assumptions related to the cost of delivery of iCBT in various scenarios, and guided 
iCBT remained the optimal strategy in all scenarios 

e The lifetime cost-effectiveness of guided iCBT with a therapist as a single therapy for the 
treatment of reccurent major depression and anxiety remains unclear. We examined the 
long-term cost-effectiveness of guided iCBT over usual care in a stepped-care modelling 
approach (secondary research question) because this approach was most clinically 
plausible, was evidence-based, and was supported by expert consultations and current 
policy development in Ontario 

e Inthe long-term lifetime model, we did not fully address the complexity of secondary 
psychiatric care, nor did we model the use of CBT in complex depression or the 
effectiveness of various additional types of treatment for treatment-resistant depression. 
Due to limitations in the existing literature, we also assumed that the efficacy of CBT is 
the same for every new episode and that there is no difference in the effectiveness of 
CBT delivery among the various types of regulated providers of psychotherapy services 

e Waiting time costs associated with in-person CBT were not included in our analyses 

e Ascenario analysis addressing a societal perspective did not include time loss and costs 
incurred by caregivers 
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Conclusions 


Compared with other available options, guided iCBT may represent an optimal and 
economically attractive alternative and could be offered as an initial step for the short-term 
treatment of eligible adults with mild to moderate major depression and/or anxiety disorders. 
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BUDGET IMPACT ANALYSIS 


We conducted a budget impact analysis to estimate the cost burden of publicly funding iCBT for 
individuals with mild to moderate major depression or anxiety disorders in Ontario over the next 
5 years. The analysis was done from the perspective of the Ontario Ministry of Health and Long- 
Term Care. All costs were reported in 2018 Canadian dollars. Reporting and analysis were done 
in accordance with the 2012 International Society for Pharmacoeconomics and Outcomes 
Research good-practice guidelines for budget impact analysis. ''? 


Research Questions 


e What is the net budget impact of publicly funding unguided or guided iCBT for adults 
with mild to moderate major depression in Ontario over the next 5 years? 


e What is the net budget impact of publicly funding unguided or guided iCBT for adults 
with anxiety disorders in Ontario over the next 5 years? 


Methods 
Analytic Framework 


We estimated the budget impact of unguided and guided iCBT interventions using the cost 
difference between two scenarios: current clinical practice without publicly funded iCBT (the 
current scenario), and the anticipated clinical practice with the publicly funded iCBT for 
individuals with mild to moderate major depression or anxiety disorders (the new scenario). All 
strategies related to current and new scenarios were described in our primary economic 
evaluation (see Table 7), where they are referred to as comparator (usual care) and intervention 
(iCBT). The model schematic is shown in Figure 9. 


We conducted a reference case analysis and several sensitivity analyses. For all analyses, we 
used outputs from our cost-effectiveness model to estimate budget impact. By doing so, we 
ensured that our budget impact analysis accounted for heterogeneity in the patient populations 
with respect to disease severity, and potential differences in response to treatments, dropout 
rates, and subsequent differences in resource use and costs. 
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Size of the Tt population 


Current Scenario New Scenario 


} Distribution of usual care without CBT } Distribution of usual care without CBT usual care without iCBT Distribution of inal treatment strategies with CBT of initial treatment Distribution of inal treatment strategies with CBT with iCBT 
Total cost of | Total costof usuatcare | care Total cost of | Totalcostoticat =| 


Budget impact (difference in costs between the two scenarios) 


Figure 9: Budget Impact Model Schematic 


Abbreviation: iCBT, internet-delivered cognitive behavioural therapy. 


Key Assumptions 


The assumptions in this analysis are described in the primary economic evaluation. In addition, 
we considered the following: 


e Our target population was based on published prevalence data; the prevalence of 
anxiety disorders in the reference case may be underestimated 

e Acertain percentage of people with substance-abuse disorders are considered eligible 
for iCBT (S. Rao, MD, and J. Hensel, MD, written communications, May 2018) 

e Eligibility of the target population was not restricted by patient readiness for iCBT; we 
assumed that people self-referred and were therefore ready and willing to try this type of 
CBT 

e Because the course of iCBT was paid upon initial sign up, public funding would still 
cover the program costs of people who did not complete the full course 


Target Population 


The target population is adults diagnosed with mild to moderate major depression or any anxiety 
disorder (as defined by DSM-5 criteria) who are eligible to use iCBT in Ontario. The lifetime 
prevalence of major depression in Canada is 10.8%; annual and 1-month prevalence estimates 
are about 4.0% and 1.3%, respectively.'7°'?3 Mild to moderate major depression occurs in 
approximately 80% of patients with prevalent major depression.’°:”6 Anxiety disorders are 
among the most common mental disorders, with lifetime prevalence rates ranging from 10.4% to 
28.8%, and 12-month prevalence rates of about 18%.°® Other Canadian data, accounting for 
differences in diagnosis of anxiety between ICD-10 and DSM-IV suggest a lower 12-month 
prevalence of any anxiety disorder of 10.2%.'4 Generalized anxiety disorder is the most 
common form, with a 12-month prevalence rate of 2.5%.'° 


Approximately 50% of people with major depression also suffer from an anxiety disorder.'° The 
percentage of coexisting anxiety disorders is even higher, with 59% to 79% of people with 
anxiety disorder having more than one. In our estimation of the target populations, we will focus 
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on the primary diagnosis of either major depression (with or without a coexisting anxiety 
disorder) or anxiety alone. 


Based on population-based data, the incidence of intellectual disability co-occurring in patients 
with a mental illness that cannot engage in CBT ranges between 3.7% and 5.2%.'*° Eligibility for 
iCBT is dependent on access to a computer. About 89% of people in Canada have access to 
computers and routinely access the internet.'*® We adjusted the population for ineligibility to 
iCBT due to learning disabilities (5%)'*° and due to a substance use disorder (10%). 


Tables 26 and 27 present the prevalence of major depression and anxiety disorders, adjusted 
for the population growth and the corresponding number of patients eligible for iCBT with major 
depression and/or an anxiety disorder. The annual number of eligible individuals with mild to 
moderate major depression ranges from about 266,400 to 277,200. 


Of note, there is no restriction in eligibility to iCBT in people with anxiety disorders with respect 
to the severity of the disease. As a result, the number of potentially eligible people with anxiety 
is much greater when compared with those with major depression. The annual number of 
potentially eligible individuals with anxiety disorders ranges from about 435,500 to 453,200. 


Table 26: Number of People With Mild to Moderate Major Depression 
(Combined With Anxiety) Eligible for iCBT in Ontario, 2018 to 2022 


No of People 
With Mild to 
No. of People Moderate Major 
Estimated No. of No. of People With Major Depression 
Adults in With Major Depression Eligible for 
Year Ontario? Depression? Eligible for CBT° iCBT? 
2018 11,287,810 440,225 333,030 266,424 
2019 11,400,688 444,627 336,360 269,088 
2020 11,514,695 449,073 339,724 271,779 
2020 11,629,842 453,564 343,121 274,497 
2021 11,746,140 458,099 346,552 277,242 


Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT. 

Assuming 1% growth in population per year (Statistics Canada). 

>Assuming major depression prevalence of 3.9%. 

°After excluding 11% with no access to the internet and 15% unable to engage in CBT because of disability or substance abuse. 
4Assuming the proportion of mild to moderate major depression is 80% of eligible cases. 
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Table 27: Number of People With Anxiety Disorder Eligible for iCBT in Ontario, 2018 to 2022 


Combined 
major No. of People __No. of People With 
Estimated No. of No. of People depression WithAnxiety Anxiety Disorder 
Adults in With Anxiety and Anxiety, Disorders Alone Eligible for 

Year Ontario# Disorders> % Only iCBT° 
2018 11,287,810 1,151,357 50 575,678 435,500 
2019 11,400,688 1,162,870 50 581,435 439,856 
2020 11,514,695 1,174,499 50 587,249 444,254 
2020 11,629,842 1,186,244 50 593,122 448,697 
2021 11,746,140 1,198,106 50 599,053 453,186 


Abbreviation: iCBT, internet-delivered cognitive behavioural therapy. 

aAssuming 1% growth in population per year (Statistics Canada). 

>Assuming prevalence of 10.2%. 

After excluding 11% with no access to the internet and 15% unable to engage in CBT because of disability or substance abuse. 


Current Intervention Mix 


We are not aware of any intervention mix of publicly funded iCBT and in-person CBT for people 
with mild to moderate major depression or anxiety disorders. We used the change in uptake of 
iCBT over time and calculated the budget impact separately for guided and unguided iCBT 
strategies. We assumed no changes in the costs of usual care with the introduction of iCBT. 


Uptake of the New Intervention and Market Effects 


In the reference case, we assumed that access to iCBT would increase by 3% each year (from 
0% to 15% over 5 years). This indicates very limited access to publicly funded iCBT at 
baseline.''® The degree of the influence of an increasing access rate over time on the net 
budget impact will be tested in our sensitivity analysis. Based on expert consultation and 
experience in other Canadian provinces, a growth of up to 25% could be expected with proper 
government funded (P. Farvolden, MD, written communication, May 2018). 


Assuming 3% access to iCBT, we estimated the number of people with major depression or an 
anxiety disorder in year 1 to be about 7,990 and 13,060, respectively (Tables 28 and 29). The 
estimate of all people at risk over a 5-year period took into account the uptake rate, prevalence 
of mild to moderate major depression/anxiety disorder in the first year, and subsequent changes 
in the distribution of eligible individuals (assuming a 15% recurrence each year'?’ accounting for 
people who become ineligible for iCBT due to disease progression). 


Ontario Health Technology Assessment Series; Vol. 19: No. 6, pp. 1-199, February 2019 107 


Budget Impact Analysis February 2019 


Table 28: Estimate of Eligible Cohort of Patients With Mild to Moderate Major Depression in 
Ontario, Years 1-5 


Cohort Estimate, 
Accounting for 


No. of People Disease 
With Mild to Progression and Target 
Moderate Major People who Had Yearly Population Target Population 
Depression iCBT in Prior Uptake of Eligible for Continuing with 

Year Eligible for iCBT Year, N@ iCBT iCBT, N Usual Care, N 
Year 1 266,424 266,424 0.03 7,993 258,431 
Year 2 269,088 220,732 0.06 13,244 255,844 
Year 3 271,779 217,768 0.09 19,599 252,180 
Year 4 274,497 213,723 0.12 25,647 248,850 
Year 5 277,242 210,009 0.15 31,501 245,740 


Abbreviations: iCBT, internet-delivered cognitive behavioural therapy. 
Assuming 15% had recurrence and progressed to more severe major depression and became ineligible for iCBT course. 


Table 29: Estimate of Eligible Cohort of Patients With Anxiety Disorders in Ontario, 


Years 1-5 
Cohort Estimate, 
Accounting for 
No. of People Disease 
With Mild to Progression and Target 
Moderate Major People who Had Yearly Population Target Population 
Depression iCBT in Prior Uptake of Eligible for Continuing with 

Year Eligible for iCBT Year, N2 iCBT iCBT, N Usual Care, N 
Year 1 435,500 435,500 0.03 13,065 422,435 
Year 2 439,856 360,812 0.06 21,649 418,207 
Year 3 444,254 355,967 0.09 32,037 412,217 
Year 4 448,697 349,355 0.12 41,923 406,774 
Year 5 453,184 343,283 0.15 51,493 401,691 


Abbreviations: iCBT, internet-delivered cognitive behavioural therapy. 
Assuming 15% had recurrence and progressed to more severe major depression and became ineligible for iCBT course. 


Future Intervention Mix 


No mix of psychological interventions is expected for the future scenario in the management of 
mild to moderate major depression or anxiety disorder. Without knowing the exact changes in 
treatment mix over time, we assumed that iCBT would not substitute for any type of in-person 
CBT in Ontario over the next 5 years and would be only an addition to usual care. 


Resource and Costs 


The costs were derived from our deterministic cost—utility analysis. As shown in Figure 9, for the 
new scenario, we assumed that the costs of iCBT varied by the type of delivery (guided vs 
unguided). The total cost estimates were based on the model outputs of undiscounted direct 
medical costs. In adults with major depression, guided (therapist) and unguided iCBT were 
associated with a mean cost of $1,666 and $275, respectively. Usual care was associated with 
a mean cost of $409. In adults with anxiety disorders, guided (therapist) and unguided iCBT 
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were associated with a mean cost of $1,665 and $270, respectively. Usual care was associated 
with a mean cost of $409. Table 30 presents the total costs associated with the future and 
current scenarios over the next 5 years for adults with mild to moderate major depression. 
Table 31 presents the corresponding costs for adults with anxiety disorders. 
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Table 30: Future and Current Scenarios, Cost Estimates Over 5 Years—Reference Case Analysis: Patients With Mild to 
Moderate Major Depression 


Future Scenario Current Scenario 


Year Guided iCBT Unguided iCBT Usual Care Guided iCBT? Unguided iCBT@ Usual Care 

Year 1 $13,317,921 $2,195,237 $105,802,833 $119,120,754 $107,998,070 $109,075,086 
Year 2 $22,067,795 $3,637,507 $104,743,714 $126,811 ,509 $108,381 ,221 $110,165,837 
Year 3 $32,657,207 $5,382,995 $103,243,523 $135,900,730 $108,626,517 $111,267,495 
Year 4 $42,734,124 $7,044,006 $101,880,269 $144,614,394 $108,924,275 $112,380,170 
Year 5 $52,489,280 $8,651,980 $100,607,200 $153,096,481 $109,259,181 $113,503,972 


Abbreviation: iCBT, internet-delivered cognitive behavioural therapy. 
*Totals for future scenarios with guided or unguided iCBT. They represent sums of data presented for guided or unguided iCBT and for usual care. 


Table 31: Future and Current Scenarios, Cost Estimates Over 5 Years—Reference Case Analysis: Patients With 
Anxiety Disorders 


Future Scenario Current Scenario 


Unguided Usual Care 
Year Guided iCBT Unguided iCBT Usual Care Guided iCBT@ iCBT@ 
Year 1 $21,755,489 $3,523,618 $172,946,846 $194,702,335 $176,470,464 $178,295,717 
Year 2 $36,048,860 $5,838,637 $171,215,659 $207,264,519 $177,054,296 $180,078,743 
Year 3 $53,347,166 $8,640,349 $168,763,323 $222,110,490 $177,403,673 $181,879,422 
Year 4 $69,808,378 $11,306,482 $166,535,072 $236,343,449 $177,841 ,554 $183,698,374 
Year 5 $85,743,881 $13,887,469 $164,454,035 $250,197,916 $178,341 ,504 $185,535,290 


Abbreviation: iCBT, internet-delivered cognitive behavioural therapy. 


Totals for future scenarios with guided or unguided iCBT. They represent sums of data presented for guided or unguided iCBT and for usual care. 
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Analysis 


To address the Ontario context for the different scenarios, we conducted the following budget 
impact analyses: 


e Reference case analysis—to estimate the net budget impact of therapist-guided iCBT 
provided by regulated health care professionals (nonphysicians) compared with usual 
care for adults with mild to moderate depression and adults with anxiety disorders 

e Reference case analysis—to estimate the net budget impact of unguided iCBT 
compared with usual care for adults with mild to moderate depression and adults with 
anxiety disorders 

e Sensitivity analysis, scenario 1—to estimate how increases in access to iCBT over 
time (i.e., increases in the uptake rate) affect the net budget impact of therapist-guided 
iCBT. The uptake rate used in this scenario was 5% in year 1, increasing 5% per year to 
25% in Year 5 (the reference case analysis assumed 3% in year 1, increasing 3% per 
year, reaching 15% in year 5) 

e Sensitivity analysis, scenario 2—to estimate the net budget impact of coach-guided 
iCBT provided by nonregulated mental health workers. The costs associated with coach- 
guided iCBT were estimated at about $1,590 per person per year (the corresponding 
cost for therapist-based iCBT used in the reference case analysis was about $1,670 per 
person per year). The costs of usual care remained the same as in the reference case 
analysis ($409 per person per year) 

e Sensitivity analysis, scenario 3—1to estimate the net budget impact of guided iCBT 
provided by regulated health care professionals, assuming no training costs. The costs 
associated with guided iCBT were estimated at about $1,087 per person per year (the 
corresponding cost for therapist-based iCBT used in the reference case analysis was 
about $1,670 per person per year). The costs of usual care remained the same as in the 
reference case analysis ($409 per person per year) 

e Sensitivity analysis, scenario 4—to estimate the net budget impact of stepped-care 
iCBT versus usual care. In this scenario, unguided was followed by therapist-guided 
iCBT for those individuals who tended to drop out from the unguided iCBT. The 
estimated annual cost of step-level iCBT was about $715 per person with mild to 
moderate major depression and about $711 per person with an anxiety disorder 

e Sensitivity analysis, scenario 5—to estimate how an increase in the prevalence of 
anxiety disorders may change the net budget impact of therapist-guided iCBT provided 
by regulated health care professionals. The prevalence of anxiety disorders used in this 
scenario was 18% and 25% 


The last four scenario analyses assume the same rate of uptake for iCBT (3% per year) as in 
the reference case analyses. Scenarios 2—4 assume the same prevalence estimates for major 
depression and anxiety disorders as in the reference case. 


All analyses were conducted using Excel 2013 (Microsoft, Redmond, WA). 


Results 
Reference Case: Budget Impact for Mild to Moderate Major Depression 


Table 32 presents calculations of the net budget impact in detail, using the data presented in 
Table 30 for the scenario comparing therapist-guided iCBT with usual care. This strategy was 
associated with the lowest ICER in our cost-utility analysis for people with mild to moderate 
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major depression. Adopting therapist-guided iCBT at a 3% uptake rate in year 1, increasing to 
15% in year 5 would lead to an increase in costs of about $10 million in year 1 to about $40 
million in year 5. 


Table 33 presents calculations of the net budget impact for the scenario comparing unguided 
iCBT with usual care in adults with mild to moderate major depression. In our cost—utility 
analysis, this strategy was associated with a smaller mean cost as compared with usual care 
but also a slight decrement in QALYs. As a result, adopting an unguided iCBT strategy at a 3% 
uptake rate in year 1, increasing to 15% in year 5 would lead to cost savings of about $1 million 
in year 1 to about $4 million in year 5. 


Reference Case: Budget Impact for Anxiety Disorders 


Table 34 presents calculations on the net budget impact in detail, using data presented in Table 
31 for the scenario comparing guided iCBT with usual care in adults with a primary diagnosis of 
anxiety disorder. In our cost—utility analysis, therapist-guided iCBT was the best strategy. 
Adopting therapist-guided iCBT at a 3% uptake rate in year 1, increasing to 15% in year 5 would 
lead to an increase in costs of about $16 million in year 1 to about $65 million in year 5. 


Table 35 presents calculations of the net budget impact for the scenario comparing unguided 
iCBT with usual care in adults with anxiety disorders. In our cost—utility analysis, this strategy 
was associated with a smaller mean cost as compared with usual care, and a slight increment in 
QALYs. As a result, adopting unguided iCBT strategy at a 3% uptake rate in year 1, increasing 
to 15% in year 5 would lead to cost savings of about $2 million in year 1 to about $7 million in 
year 5. 


Table 32: Net Budget Impact of Adopting Therapist-Guided iCBT in Ontario for Adults With Mild to 
Moderate Major Depression: Years 1-5 


Scenario Year 1 Year 2 Year 3 Year 4 Year 5 Total 

Usual care $109,075,086 $110,165,837 $111,267,495 $112,380,170 $113,503,972 $556,392,560 
Guided iCBT $119,120,754 $126,811,510 $135,900,730 $144,614,394 $153,096,481 $679,543,869 
NBI $10,045,669 $16,645,673 $24,633,235 $32,234,224 $39,592,509 $123,151,309 


Abbreviations: iCBT, internet-delivered cognitive behavioural therapy; NBI, net budget impact. 


Table 33: Net Budget Impact of Adopting Unguided iCBT in Ontario for Adults With Mild to 
Moderate Major Depression: Years 1-5 


Scenario Year 1 Year 2 Year 3 Year 4 Year 5 Total 

Usual care $109,075,086 $110,165,837 $111,267,495 $112,380,170  $113,503,972 $556,392,560 
eee $107,998,070 $108,381,221 $108,626,517 $108,924,275 $109,259,181 $543,189,265 
NBI -$1,077,016 -$1,784,615 -$2,640,978 -$3,455,895 -$4,244,791 -$13,203,295 


Abbreviations: iCBT, internet-delivered cognitive behavioural therapy; NBI, net budget impact. 
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Table 34: Net Budget Impact of Adopting Therapist-Guided iCBT in Ontario for Adults With 
Anxiety Disorders: Years 1-5 


Scenario Year 1 Year 2 Year 3 Year 4 Year 5 Total 
Usual care $178,295,718 $180,078,743  $181,879,422 $183,698,374 $185,535,290 $909,487,546 
Guided iCBT $194,702,335  $207,264,519  $222,110,490 $236,343,449 $250,197,916 $1,110,618,710 
NBI $16,406,618 $27,185,777 $40,231,068 $52,645,075 $64,662,627 $201,131,165 


Abbreviations: iCBT, internet-delivered cognitive behavioural therapy; NBI, net budget impact. 


Table 35: Net Budget Impact of Adopting Unguided iCBT in Ontario for Adults With Anxiety 
Disorders: Years 1-5 


Scenario Year 1 Year 2 Year 3 Year 4 Year 5 Total 
Usual care $178,295,718 $180,078,743 $181,879,422 $183,698,374 $185,535,290 $909,487,546 
Unguided $176,470,464  $177,054,297 $177,403,673 $177,841,554  $178,341,504 $887,111,492 
iCBT 

NBI -$1,825,254 -$3,024,446 -$4,475,749 -$5,856,820 -$7,193,785 -$22,376,054 


Abbreviations: iCBT, internet-delivered cognitive behavioural therapy; NBI, net budget impact. 


Sensitivity Analysis 


Scenario 1: Budget Impact for Major Depression and Anxiety Disorders Assuming 
Larger Access to Therapist-Guided Internt CBT Over Time 


In adults with mild to moderate major depression, therapist-guided iCBT at a 5% uptake rate in 
year 1, increasing to 25% in year 5, would lead to costs of about $17 million in year 1, 
increasing to about $61 million in year 5 (Table 36). 


In with anxiety disorders, therapist-guided iCBT at a 5% uptake rate in year 1, increasing to 25% 
in year 5, would lead to costs of about $27 million in year 1, increasing to about $100 million in 
year 5 (Table 37). 


Table 36: Net Budget Impact of Adopting Therapist-Guided iCBT in Ontario for Adults With Mild to 
Moderate Major Depression, Increase in Access by 5% Per Year: Years 1-5 


Scenario Year 1 Year 2 Year 3 Year 4 Year 5 Total 

Usual care $109,075,086 $110,165,837 $111,267,495  $112,380,170 $113,503,972  $556,392,560 
Guided iCBT $125,817,867 $137,238,914  $150,758,776 $163,132,267 $174,862,019 $751,809,843 
NBI $16,742,781 $27,073,077 $39,491,281 $50,752,097 $61,358,047 $195,417,283 


Abbreviations: iCBT, internet-delivered cognitive behavioural therapy; NBI, net budget impact. 
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Table 37: Net Budget Impact of Adopting Therapist-guided iCBT in Ontario for Adults With Anxiety 
Disorders, Increase in Access by 5% per year: Years 1-5 


Scenario Year 1 Year 2 Year 3 Year 4 Year 5 Total 


Usual care $178,295,718 $180,078,743  $181,879,422 $183,698,374 $185,535,290 $909,487,546 
Guided iCBT  $205,640,081  $224,294596  $246,376,690  $266,586,926 $285,745,465  $1,228,643,758 


NBI $27,344,363 $44,215,853 $64,497,268 $82,888,552  $100,210,175 319,156,212 


Abbreviations: iCBT, internet-delivered cognitive behavioural therapy; NBI, net budget impact. 


Scenario 2: Budget Impact for Major Depression and Anxiety Disorders Assuming 
Coach-Delivered Guided iCBT 


In adults with mild to moderate major depression, adopting coach-guided iCBT at a 3% uptake 
rate per year would lead to costs of about $9 million in year 1, increasing to about $37 million in 
year 5 (Table 38). 


In adults with anxiety disorders, adopting coach-guided iCBT at a 3% uptake rate per year 
would lead to costs of about $15 million in year 1, increasing to about $61 million in year 5 
(Table 39). 


Scenario 3: Budget Impact for Major Depression and Anxiety Disorders for 
Guided iCBT Assuming No Additional Spending on Therapist Training 


In adults with mild to moderate major depression, adopting therapist-guided iCBT at a 3% 
uptake rate per year (whereby the course of iCBT excluded the training costs), would lead to 
costs of about $5 million in year 1, increasing to about $21 million in year 5 (Table 40). 


In adults with anxiety disorders, adopting therapist-guided iCBT at a 3% uptake rate per year 
(the course of inernet CBT excludes training costs), would lead to costs of about $9 million in 
year 1, increasing to about $35 million in year 5 (Table 41). 


Table 38: Net Budget Impact of Adopting Coach-Guided iCBT in Ontario for Adults With Mild to 
Moderate Major Depression: Years 1-5 


Scenario Year 1 Year 2 Year 3 Year 4 Year 5 Total 

Usual care $109,075,086 $110,165,837 $111,267,495 $112,380,170  $113,503,972 $556,392,560 
GuidediCBT  $118,509,898  $125,799,321  $134,402,835  $142,654,298  $150,688,943 $672,055,296 
NBI $9,434,812 $15,633,484 $23,135,340 $30,274,128 $37,184,972 $115,662,736 


Abbreviations: iCBT, internet-delivered cognitive behavioural therapy; NBI, net budget impact. 


Table 39: Net Budget Impact of Adopting Coach-Guided iCBT in Ontario for Adults With Anxiety 
Disorders: Years 1-5 


Scenario Year 1 Year 2 Year 3 Year 4 Year 5 Total 
Usual care $178,295,718 $180,078,743 $181,879,422 $183,698,374  $185,535,290  $909,487,546 
Guided iCBT $193,710,097 $205,620,379 $219,677,397  $233,159,583  $246,287,253  $1,098,454,710 
NBI $15,414,379 $25,541,637 $37,797,975 $49,461,209 $60,751,963 $188,967,164 


Abbreviations: iCBT, internet-delivered cognitive behavioural therapy; NBI, net budget impact. 
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Table 40: Net Budget Impact of Adopting Guided iCBT (Assuming No Additional Costs on 
Therapist Training) in Ontario for Adults With Mild to Moderate Major Depression: 


Years 1-5 
Scenario Year 1 Year 2 Year 3 Year 4 Year 5 Total 
Usual care $109,075,086 $110,165,837 $111,267,495  $112,380,170 $113,503,972  $556,392,560 
Guided iCBT $114,486,833  $119,133,102 $124,537,776 $129,745,214  $134,833,031  $622,735,958 
NBI $5,411,747 $8,967,266 $13,270,281 $17,365,044 $21,329,059 $66,343,398 


Abbreviations: iCBT, internet-delivered cognitive behavioural therapy; NBI, net budget impact. 


Table 41: Net Budget Impact of Adopting Guided iCBT (Assuming No Additional Costs on 
Therapist Training) in Ontario for Adults With Anxiety Disorders: Years 1-5 


Scenario Year 1 Year 2 Year 3 Year 4 Year 5 Total 
Usual care $178,295,718 $180,078,743 $181,879,422  $183,698,374 $185,535,290 $909,487,546 
GuidediCBT  $187,145,381  $194,742,641 $203,579,898  $212,094,915 $220,414,050  $1,017,976,885 
NBI $8,849,664 $14,663,898 $21,700,476 $28,396,541 $34,878,760 $108,489,339 


Abbreviations: iCBT, internet-delivered cognitive behavioural therapy; NBI, net budget impact. 


Scenario 4: Budget Impact for Stepped-Care Delivery of Unguided and Therapist- 
Guided iCBT for Major Depression and Anxiety Disorders 


In adults with mild to moderate major depression, adopting unguided followed by therapist- 
guided iCBT via a stepped-care approach (at a 3% uptake rate per year) would lead to costs of 
about $2.5 million in year 1, increasing to about $10 million in year 5 (Table 42). 


In adults with anxiety disorders, adopting unguided followed by therapist- guided iCBT via a 
stepped-care approach would lead to costs of about $4 million in year 1, increasing to about 
$16 million in year 5 (Table 43). 


Scenario 5: Budget Impact Associated With Higher Estimates of Prevalence of 
Anxiety Disorders 


Assuming a prevalence of 18% for anxiety disorders, adopting therapist-guided iCBT at a 3% 
uptake rate per year would lead to costs of about $28 million in year 1, increasing to about 
$114 million in year 5 (Table 44). Assuming a prevalence of 25% for anxiety disorders, adopting 
therapist-guided iCBT would lead to an increase in costs of about $40 million in year 1 to 

$158 million in year 5 (a total 5-year net budget impact of $493 million, data not shown). 
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Table 42: Net Budget Impact of Adopting Guided iCBT via a Stepped-Care Delivery in Ontario for 
Adults With Mild to Moderate Major Depression: Years 1-5 


Scenario Year 1 Year 2 Year 3 Year 4 Year 5 Total 
Usual care $109,075,086 $110,165,837 $111,267,495 $112,380,170 $113,503,972 $556,392,560 
Unguided $111,520,903 $114,218,556  $117,264,946  $120,228,232  $123,143,555 $586,376,192 
followed 

by guided 

iCBT 

NBI $2,445,818 $4,052,720 $5,997,450 $7,848,062 $9,639,583 $29,983,633 


Abbreviations: iCBT, internet-delivered cognitive behavioural therapy; NBI, net budget impact. 


Table 43: Net Budget Impact of Adopting Guided iCBT via a Stepped-Care Delivery in Ontario for 
Adults With Anxiety Disorder: Years 1-5 


Scenario Year 1 Year 2 Year 3 Year 4 Year 5 Total 
Usual care $178,295,718 $180,078,743  $181,879,422 $183,698,374 $185,535,290 $909,487,546 
Unguided $182,231,105  $186,599,682 $191,529,481  $196,326,130  $201,045,646 $957,732,045 
followed 

by guided 

iCBT 

NBI $3,935,387 $6,520,940 $9,650,059 $12,627,756 $15,510,356 $48,244,499 


Abbreviations: iCBT, internet-delivered cognitive behavioural therapy; NBI, net budget impact. 


Table 44: Net Budget Impact of Adopting Guided iCBT Assuming Higher Prevalence of Anxiety 
Disorders: Years 1-5 


Scenario Year 1 Year 2 Year 3 Year 4 Year 5 Total 
Usual care $314,639,702 $317,786,090 $320,964,068  $324,173,638 $327,415,417  $1,604,978,915 
GuidediCBT  $343,592,576  $365,761,000 $391,960,159 $417,076,713  $441,526,013 $1,959,916,460 
NBI $28,952,874 $47,974,910 $70,996,091 $92,903,075 $114,110,596 $354,937,546 


Abbreviations: iCBT, internet-delivered cognitive behavioural therapy; NBI, net budget impact. 


Discussion 


We conducted a model-based budget impact analysis to examine the range of investment 
needed to enable access to iCBT for adults with a primary diagnosis of mild to moderate major 
depression or anxiety disorders, as defined by the DSM-5 criteria. 


Assuming a 3% increase in access per year (from 0% at baseline to 15% in year 5) and a target 
population increase from about 8,000 in year 1 to about 32,000 in year 5, we estimated the net 
budget impact of publicly funding guided iCBT with a therapist for the treatment of mild to 
moderate major depression would be about $10 million in year 1, increasing to about $40 million 
in year 5. The latter estimate of budget impact corresponds to treating about 12% of people with 
mild to moderate major depression in 2021 (year 5). The net budget impact would be about 

$16 million in year 1, increasing to about $65 million in year 5 for the treatment of anxiety 
disorders (the target population of 13,000 in year 1 would increase to 52,000 in year 5, 
corresponding to about 12% of people with anxiety disorders in 2021). As expected, increases 
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in the access over time or prevalence of the disease were associated with greater annual 
budget impact estimates. 


Public funding for unguided iCBT would be associated with cost savings in both populations. 
However, the budget impact analysis was model-based. It adopted the same parameter 
assumptions as were used for our cost-effectiveness analysis. Thus, unguided iCBT was 
associated with smaller total costs than usual care because of the smaller probability of 
medication use in the CBT strategies. Nevertheless, unguided iCBT is not a preferred option 
because it has relatively high dropout rates and there is a lack of evidence with respect to 
incremental clinical benefits compared with usual care. 


Interestingly, using a stepped-care model for iCBT (i.e., guided iCBT only in individuals who are 
considered at risk of dropping out from the unguided program) represented the most affordable 
option for Ontario. Over the next 5 years, the annual net budget impact ranged from about 

$2 million in year 1 to about $10 million in year 5 in adults with mild to moderate depression, 
and from about $4 million to about $16 million over the same time period in adults with anxiety 
disorders. Although this modeling approach may seem to be the most affordable option for 
Ontario, we found limited evidence of effectiveness or cost-effectiveness with unguided iCBT. 
Consequently, this analysis indicates the importance of individuals’ readiness for iCBT, which 
should be carefully assessed and considered in the implementation program. 


It is important to recognize that guided iCBT was assumed to be delivered by any regulated 
registered health care professional providing CBT for depression or anxiety and that the 
projected costs of guided therapy do not include and costs for training. However, the cost of 
iCBT includes potential maintenance costs of the portal through which e-telehealth therapy 
could be delivered. Types of professionals providing guided iCBT widely range across the 
countries. Some countries, such as the United Kingdom, have organized mental health 
programs (i.e., the UK IAPT’?®°"'°) in which internet and in-person CBT are provided by highly 
trained non-regulated mental health workers called coaches, who are supervised by regulated 
health care professionals. 


When we explored this scenario for Ontario, we found relatively small decreases in the annual 
net budget impact. If coaches deliver guided iCBT (instead of regulated registered 
psychotherapists), the savings will range between $1 million and $3 million per year (e.g., 

$9 million vs $10 million in year 1, increasing to $37 million vs $40 million in year 5 for major 
depression). We also find that guided iCBT would be more affordable if no additional costs are 
spent on therapist training (e.g., $5 million in year 1, increasing to $21 million in year 5 for major 
depression). 


Our budget impact analysis considered that, regardless of prior disease history, anyone 

diagnosed with mild to moderate major depression or anxiety disorder at the time of entrance to 
the publicly available mental health program would have only one full course of iCBT. After that, 
depending on the disease progression, individuals could continue with other forms of treatment. 


Conclusions 


Over the next 5 years, the annual net budget impact of publicly funding guided iCBT in Ontario 
would be between $10 million and $40 million for the treatment of mild to moderate major 
depression, and between $16 million and $65 million for the treatment of anxiety disorders. A 
relatively small cost savings may be achieved if guided iCBT is provided by a coach (i.e., non- 
regulated mental health worker), while moderately high cost savings may be achieved if no 
additional spending is required for therapist training or if the delivery of guided iCBT is 
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organized through a stepped-care iCBT model or is provided to a selected population who is 
ready to comply with therapy. 
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PATIENT PREFERENCES AND VALUES 


Objective 


The objective of this analysis was to explore the underlying values, needs, impacts, 
preferences, and perceptions of iCBT among people with lived experience of depression or an 
anxiety disorder. 


Background 


Patient, caregiver, and public engagement provides a unique source of information about 
people’s experiences of a health condition and the health technologies or interventions used to 
manage or treat that health condition. This information includes the impact of the condition and 
its treatment on the patient, the patient’s family and other caregivers, and the patient’s personal 
environment. Engagement also provides insights into how a health condition is managed by the 
province’s health system. 


Information shared by people with lived experience can also identify gaps or limitations in 
published research (e.g., sometimes typical outcome measures do not reflect what is important 
to those with lived experience).'7&'% Additionally, lived experience can provide information and 
perspectives on the ethical and social values implications of health technologies or 
interventions. 


Because the needs, priorities, preferences, and values of those with lived experience in Ontario 
are often not adequately explored in published literature, we contact and speak directly with 
people who live with a given health condition, including those who may have experience with 
the intervention we are exploring. 


Methods 
Engagement Plan 


The engagement plan for this health technology assessment focused on examining the 
experiences of people with depression or anxiety and those of their family, including their 
understanding and perceptions of iCBT. We engaged people via face-to-face and telephone 
interviews. 


We used a qualitative interview, as this method of engagement allows us to explore the 
meaning of central themes in the experiences of people with anxiety or depression, as well as 
those of their family. Our main task in interviewing is to understand what people tell us and to 
gain an understanding of the meaning of their experiences. '*4 The sensitive nature of exploring 
people’s experiences of a health condition and their quality of life are other factors that 
supported our choice of an interview methodology. 


Participant Outreach 


We used an approach called purposive sampling, '*° which involves actively reaching out to 
patients, families, and caregivers with direct experience of the health condition and health 
technology or intervention being reviewed. We approached more than 30 organizations and 
groups involved in providing iCBT or providing care for anxiety or depression using other 
treatments to spread the word about this engagement opportunity. 
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Inclusion Criteria 


We sought to speak with people who had been actively managing depression or anxiety with 
iCBT or another type of treatment. 


Exclusion Criteria 


We did not set specific exclusion criteria. 


Participants 


For this project, we spoke with 17 people with depression or anxiety living across Ontario, as 
well as 1 family member of people with depression or anxiety. Participants varied in terms of 
socioeconomic background, gender, and language preference. 


We spoke with people who had experience with iCBT, as well as with other treatments such as 
medication, in-person CBT, and self-help. The people with anxiety or depression with whom we 
spoke fell into two groups: one who had undergone treatment with iCBT and another who were 
interested in trying iCBT but had not yet done so. 


Fourteen participants had direct experience with iCBT, and all participants had experience with 
medication. All participants had lived experience of depression or anxiety and with treatment for 
their condition. Participants shared their experiences and perceptions either in person or over 
the telephone. 


Gaining an understanding of the day-to-day functioning of people with depression or anxiety and 
their experiences with available treatments, including iCBT, helped us assess the potential 
value of iCBT from the perspective of people living with depression or anxiety and their family 
members. 


Approach 


At the beginning of the interview, we explained the role of Health Quality Ontario, the purpose of 
this health technology assessment, the risks of participation, and how participants’ personal 
health information would be protected. We gave this information to participants both verbally 
and in a letter of information (Appendix 10). We obtained participants’ verbal consent before 
starting the interview. With participants’ consent, we audio-recorded and then transcribed the 
interviews. 


Interviews lasted about 20 to 40 minutes. Interviews were semistructured and consisted of a 
series of open-ended questions. Questions were based on a list developed by the Health 
Technology Assessment International Interest Group on Patient and Citizen Involvement in 
Health Technology Assessment.'?° Questions focused on the impact of depression or anxiety on 
participants’ quality of life and their perceptions of the benefits and limitations of various 
treatment options for managing their condition. See Appendix 11 for our interview guide. 


Data Extraction and Analysis 


We used a modified version of a grounded-theory methodology to analyze interview 

transcripts. '9”'3° The grounded-theory approach allows us to organize and compare information 
across participants. This method consists of a repetitive process of obtaining, documenting, and 
analyzing responses while simultaneously collecting, analyzing, and comparing information. We 
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used the qualitative data analysis software program NVivo (QSR International, Doncaster, 
Victoria, Australia) to identify and interpret patterns in interview data. The patterns we identified 
then allowed us to highlight the impact of depression or anxiety on the people we interviewed. 


Results 


The people with depression or anxiety and the family members with whom we spoke 
emphasized the constant struggle of managing the condition. They reported medications were a 
widely available treatment option, but although medications helped manage symptoms, they 
also had side effects that could be severe at times. In addition, all interviewees expressed that 
medications do not address the underlying cause of a person’s depression or anxiety. People 
with depression or anxiety reported being interested in exploring treatment options in addition to 
or as an alternative to medication to manage their condition in the long term. Some participants 
reported experience with in-person CBT or self-help methods in addition to taking medication. 
Some participants reported one or many socioeconomic barriers to accessing treatment. 


People with experience of iCBT were able comment on the similarities and differences of this 
treatment compared with in-person CBT. They commented that iCBT was appealing as it 
provided control over the location, time, and pace of therapy. Some people stated that the lack 
of face-to-face interaction with one’s iCBT therapist was difficult. Some also noted that despite 
learning techniques to manage their condition, they sometimes fell back into their old patterns of 
thinking and reported needing peer support and follow-up after completing therapy. 


Day-to-Day Impact of Depression or Anxiety 


Participants noted that their depression or anxiety had an immense impact on their day-to-day 
life. Most interviewees discussed struggles in the daily management of their condition, as well 
as challenges associated with low self-esteem, maintaining personal relationships, and financial 
difficulties. 


Struggles in the Day-to-Day Management of Depression or Anxiety 


People with depression or anxiety expressed difficulty understanding their condition and with 
seeking help. They mentioned that, following diagnosis, they experienced a consistent struggle 
to cope with their condition. They often felt lethargic from the effort it took to manage their 
emotions on a daily basis: 


The first challenge that | faced in terms of living with anxiety was, sort of like, 
understanding it myself. 


| would get home from work, when | was still working, be exhausted, have to lie down, 
couldn‘ start to prepare meals until I'd had a rest. 


| would need to sedate myself to be able to open the door and get outside, and | would 
have to make numerous attempts to just try to open the door. So that one thing could 
take me hours to be able to get to the mailbox and back, and it would take everything 
out of me that | had. 


Participants often attributed other health problems, such as high blood pressure, blackouts, 
panic attacks, insomnia, and lethargy, to unmanaged depression or anxiety: 
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| also lost my driver's licence because...of the blackouts. | think | was falling asleep, 
personally, just exhaustion from trying to cope with this depression. 


| believe the blood pressure was the result of the anxiety. 


[I] went to the hospital, the emergency department, many, many times with chest pain, 
so then we got into exploring did | have a heart problem. So, one thing led to another. 
As | look back, as I’ve improved, all of those health problems, | think, were a problem 

of...the anxiety and the depression. 


| have no deep sleep. I’ve had sleep studies done; | have no deep sleep; I'm waking 
every 2 to 3 minutes because of the hypervigilance. So, | have no sleep. 


It [depression] also affects our sleep. We don't sleep, and not having enough sleep 
also...just makes everything worse. 


Participants noted that depression or anxiety caused them to lose interest in other aspects of life 
such as meal preparation, managing finances, taking part in recreational activities, and 
travelling. They noted that things that had previously interested them were no longer appealing, 
with one person referring to this feeling as “drifting”: 


It has kind of affected my desire to travel and enjoy some of the things that | have 
enjoyed in the past. 


Mentally it affected my memory. | could not sit down and play the piano; | couldn’t 
organize my finances; | couldn’t balance my bank accounts or my credit card 
statements or whatever. At home, | would drift from room to room, incapable of actually 
performing a task. But then, | just...l’m generally a very good housekeeper, a good 
cook as well, [and] | like to bake. So, none of those hobbies interested me, and, of 
course, | just didn’t seem to have the energy to do any of those things. 


| would buy a ticket for a concert and one of my friends would come to pick me up, and 
! couldn't go; just, | just couldn't go. 


Low Self-Esteem 


Participants reported feeling isolated, stigmatized, and disconnected from other people, 
especially those who did not understand their condition. One person commented that their 
condition was a “huge hit” to their self-esteem and sense of self-worth. Participants commented 
that low self-esteem and a feeling of isolation also made it difficult to seek help for their 
condition: 


And there are people in my life who don’t understand mental illness, or they're 
dismissive of it. And that, of course, impacts my relationship with them, because this 
has become such a huge part of my day-to-day life, living and trying to cope with this 
condition. 


At my lowest point, | definitely felt utterly alone. | felt like | was the only person in the 
world going through this, and | couldn't access all the resources | felt that | needed. 
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You know, | have moments where | definitely still doubt myself, and even my closest 
relationships with friends and family. 


You feel very isolated; you feel like a total “nut bar.” So, when you meet people, 
doctors and nurses and other kinds of professionals, [and] there is no one [who] 
understands your suffering...words just don't give it justice. 


My relationship with myself is impacted...because | know | could be doing so much 
better and living a more wholesome life and doing [many] more things. So, it affects 
you 360 degrees all around your life. 


Managing Personal Relationships 


People with depression or anxiety mentioned that their condition also impacted their personal 
relationships. They noted that they often found it challenging to communicate with people about 
their condition: 


One of the bigger challenges now is...communicating that with other people. Because 
it is something that people are going to experience tangentially through you, you know, 
like it’s never going to be a thing that people don’t notice, or people don't see, and it’s 
going to always be something that (they) feel. 


Some interviewees noted that their condition caused them to become increasingly dependent 
on family and friends. They noted that this caused stress for their families, and that their 
condition limited their ability to take part in daily activities and travel: 


Without the friends, | possibly might have committed suicide when my husband wasn’t 
home; it had gotten quiet, quite frightening. So, it got to the point where my husband, of 
course, was very fearful and concerned; he became overprotective, constantly asking 
me how | was. [He] didn’t like me to go anywhere by myself because he was afraid | 
might have one of these episodes...[! was] pretty much withdrawn from social activities, 
especially if it was about me going anywhere, and basically | rely on my husband, 
heavily, when he’s home...to do everyday activities together such as groceries or any 
kind of thing, and [we] pretty much [stay] at home, for the most part. Not really going 
anywhere. 


Financial Impact 


Some participants whose depression or anxiety had begun early in their lives mentioned that 
their condition had impacted their ability to complete their education and find a job: 


| felt like my life had been derailed because of my anxiety disorder. It took me an extra 
year to complete my undergraduate degree, which obviously came with added financial 
stress. 


Because my agoraphobia started when | was 12, | never finished high school... 
because | couldn't leave the house. So, you can imagine a young person [who] has not 
finished school because they can't leave the house, what kind of an income that they 
would be making; it would be virtually nothing...It affected everything. 
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Participants whose depression or anxiety began later in life reported that the condition impacted 
their ability to be productive at work and stay employed. All participants noted some impact on 
their finances owing to their condition: 


Well, we’re [people with depression or anxiety] often really poor. We can’t get a 
job...For a brief period of time, | was on disability myself, so | know what that’s like, and 
it feels hopeless. You don’t have any means or ability to earn an income; you barely 
make it. 


My work was severely affected; | was exhausted all the time. Normally I’m a very 
organized, prepared person, [and] my work is up to date. But as | became more and 
more depressed, | became disorganized, | lost important papers, and | couldn’t get my 
work done on time. 


Experience with Medication and In-Person Cognitive Behavioural Therapy 


Participants reported having tried various treatment options, including medication and in-person 
CBT. People reported that medication was the most accessible treatment option. 


Medication 


Participants mentioned that medications were helpful in managing the severity of their condition 
and stabilizing their mood: 


Medication helps, and it reduces the impact when you are suffering from anxiety. 


One participant used the metaphor of a “fire in my house” to explain the severity of her 
condition. She stated that she needed to “put out the fire in my house before | can stand back 
and analyze why | might have had a fire in my house,” referring to the importance of using 
medication to regulate her mood before pursuing other types of treatment. 


However, some participants mentioned that medications sometimes only partially helped 
alleviate symptoms: 


! felt like they [medications] met my needs somewhat, in terms of helping with the mood 
and anxiety. You know, obviously, | still wasn’t at top mental and emotional shape, but | 
was definitely able to function better in day-to-day life in terms of attending classes and 
things like that. 


Many participants described medications as “hit or miss” in terms of their ability to improve 
symptoms. They mentioned having to experiment with a variety of medications before finding 
one or more that they perceived as having a positive impact on their mood. Some participants 
felt that medication alone, without also pursuing another type of treatment, did not help: 


I've tried. In the last 10 or 15 years, I've tried all kinds of different antidepressants, but 
nothing helps. 


Medication Side Effects 


Participants noted a range of medication side effects, including weight gain and an urge to cut 
oneself. They also noted having to take additional medications to manage their side effects: 
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| tried many, many drugs, some with some really bad side effects...and that’s why | had 
to change so often. 


In terms of side effects...it took a long time to find [the] right meds. | went through a 
whole lot. Some of these meds cause some really severe side effects [including] 
nausea [and] constipation, and | would have to, in turn, go and get over-the-counter 
medication to help with those side effects, if you know what | mean. So, it’s just 
medication piling up, piling up on each other. So that wasn't good. 


It got so bad once that | went to our hospital’s emergency mental health crisis centre, 
and at that time | was told that | was [taking] a drug that was making things worse, | 
think, than better. 


One family member also reported on the negative impact medication can have on a person: 


| was strongly against it [medication]...She insisted on taking it. She tried it and started 
cutting herself in 3 weeks. | saw the marks and inquired. She told me, and | asked her 
to stop the meds. 


Barriers to Accessing Medication 


Participants reported experiencing barriers to therapy with medication related to cost and 
access. 


The ongoing cost of medications was reported as a barrier for people without drug coverage or 
living on a fixed income. Most interviewees with depression or anxiety reported that the cost of 
medication was a substantial barrier: 


| felt like, for me, again, cost was another factor in terms of pharmacotherapy, the 
medication. Again, | was not covered on an insurance plan and had to rely on the 
Trillium public drug plan, which...in my case had pretty high deductibles. 


I’m living in poverty, and...my husband has mental health issues as well, so that again 
impacts me because of his issues. So basically, we just talk because | have a lot of 
stresses in my life that | can’t do anything about. | can’t do anything about the poverty. 


Some participants, especially those from Northern Ontario, noted that it was difficult for them to 
receive a diagnosis and to access mental health care: 


Well, it was very, very difficult here [in Northern Ontario]...to get a psychiatrist, but my 
doctor recognized that; he...attempted to find the right mix [of medication and/or 
treatment] that would work for me, but we’re very short of psychiatrists. 


Because neither family doctors nor [we], the general public, are aware of places where 
you can go for help or get treatment and how you get involved in it, you know. So, there 
is a disconnect somewhere between the services that are available and the public and 
even doctors working within the health care system; they don’t know about everything 
that’s there. 
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Alternatives to Medication 


Many participants reported seeking out treatments as an alternative or in addition to medication. 
Some mentioned that medication helped alleviate symptoms during an episode of anxiety or 
depression; however, they felt medication was not a long-term solution for their condition. Some 
also noted feeling “trapped” by their medications. They expressed interest in developing ways to 
manage their condition without having to rely solely on their medications. Several interviewees 
also reported trying self-help methods such as relaxation therapy, meditation, and self-help 
books to supplement their medication or while they were waiting to receive therapy such as 
CBT: 


| do have friends who have tried...meds, and | feel like it doesn’t necessarily...address 
the root of the problem. 


When you have really bad anxiety like | do, you dont want to be under the control of 
drugs either, so pharmaceuticals were really not much of an option for me, because | 
felt equally as trapped taking drugs as | did with the anxiety itself. So, | tried all kinds of 
things, but | didn’t stick with very many things for that reason. 


In-Person Cognitive Behavioural Therapy 


For the purposes of this project, we were interested in comparing participants’ experiences with 
in-person CBT versus iCBT. Participants reported finding in-person CBT helpful in improving 
their mood and anxiety but that it was a challenging treatment to undergo: 


| have found cognitive behavioural therapy to be...challenging, because it requires a lot 
of...interpersonal reflection and...thinking, and it can be really...emotionally 
exhausting. 


There’s a great deal of work that has to go into helping yourself get better; nobody can 
do it for you. You’ve got to put a lot of...work in it, and [you've] got to be really 
determined to improve, that you don’t want your life to be like this anymore. 


One-on-One Versus Group Sessions 


Participants who had experience with both one-on-one and group therapy sessions reported 
that each has its own benefits and challenges. They noted that one-on-one sessions allow for a 
more personalized approach to therapy, whereas group sessions provide peer support and a 
chance to relate with others with similar issues. However, group therapy does not provide the 
privacy some felt is necessary for sharing personal information. Some reported that this lack of 
privacy limited their ability to work on certain issues: 


Because it is a group session, you are not given as much personal care. Time is 
allocated between people. Even though after weeks you are more at ease...you are 
still not as transparent [as] if you are in [a] private session. 


Barriers to Accessing In-Person Cognitive Behavioural Therapy 
Despite participants’ interest in receiving in-person CBT, many faced barriers of cost and 


access to this type of therapy. Most participants found they were often unable to afford the cost 
of in-person CBT: 
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What I really wanted was somebody to...help me in a nonpharmaceutical way, but.../ 
didn’t have the financial means to be able to hire a psychologist or anybody [who] 
would be able to provide me with any help 


| wasn't covered by insurance, and the psychologist [who] | would reach out to or [had] 
researched, their average starting rate was something like $200 an hour, which 
obviously, for a student, you can’t really afford. 


Some participants also mentioned the costs associated with travel to and from CBT sessions: 


| definitely had to pay for parking and sometimes get a ride, ask a relative to give me a 
lift. 


Some of those unable to access in-person CBT were advised by their physicians to visit the 
emergency department to receive care in addition to medication: 


And the person that she [physician] spoke with...said, “You know, there are no 
community psychologists covered by OHIP...You’d be better off sending your patient to 
the emergency, saying that they are a suicide risk.” And that is the way that you will get 
services. 


Limitations of In-Person Cognitive Behavioural Therapy 


Participants reported several limitations to in-person CBT, including the time commitment 
required, the number of sessions included in a course of CBT, and the lack of follow-up support. 
They also reported that the severity of one’s symptoms could make it difficult to get to therapy 
sessions. 


Time Commitment 


Participants with busy schedules found it challenging to accommodate therapy appointments 
and travel: 


He [psychiatrist], in fact, was the one who suggested that | take a CBT course for this 
[anxiety], but since | work full-time, | was not able to choose any timing during my work 
hours, obviously. 


Number of Sessions in a Course of In-Person CBT 


Some participants mentioned that a single 8- to 12-week course of in-person CBT was 
sometimes insufficient. They reported sometimes struggling to incorporate the skills they had 
learned into their daily lives and that refresher sessions were needed to maintain their skills. 
One patient reported the perception that, in a 10-week course, a person usually gets less than 5 
hours of actual therapy: 


| guess another sort of drawback would be, like, it was an eight-week-long session, and 
| felt that it’s really towards the end that you sort of start experiencing the benefits of 
those sessions, and then it sort of ended. 


She [therapist] gets 10 sessions with one [client], so the first four or five [sessions] she 
can do planning work, and then she gets four or five sessions to do CBT... You may get 
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40 minutes with them, and to do four or five sessions of 40 minutes of CBT, that’s just 
useless...Less than five hours of CBT with a social worker: they're [clients] not going to 
learn anything. 


Follow-Up Support 


Participants mentioned that even though they had learned coping skills to manage their 
symptoms through CBT, a high-stress environment or change of routine after completing 
therapy could cause them to revert to their previous habits of thinking. Participants felt that 
following a routine and practising self-care helped them manage their condition. A few people 
mentioned peer support following therapy might help them stay accountable and reinforce the 
skills learned in CBT. It was mentioned that acting as a peer support might also assist in a 
person’s recovery through a change in mindset from “victim” to “helping others”: 


Because the way you deal with anxiety and stuff [depression], those are all skills that you 
have to learn, and you have to practise just like any other skill. And if you get really busy, 
just like any other skill, it’s something that kind of takes the back burner a little bit. And it 
takes a little bit of time to [try] to get back on the horse and like be, like, “Okay, | have to 
remember that | have to be doing this and managing my thinking and managing...those 
thoughts.” 


| have a certain routine that | stick to that | know...lends to me being more calm and 
confident and capable. So, if | have to make, for example, medical appointments or 
things | have to get to, either before or after work, | find that really challenging or 
frustrating. Or, it definitely throws me off and makes me feel more anxious and 
frustrated, and that becomes difficult. 


They [therapists] give tools and exercises and avenues to explore. After 8 weeks [of 
therapy], you feel that you start back in the loop of [a] depressive phase. You are not 
using the exercises in the more scheduled manner anymore. 


Nature of Condition 


Some interviewees reported finding it difficult to attend in-person therapy, owing to the nature of 
the symptoms of depression and anxiety. This theme was especially common for people with 
anxiety and those who were homebound owing to the severity of their condition: 


...a lot of people with anxiety or agoraphobia...can’t get treatment...can’t even get there, 
we cant even leave our houses, so even if there were centres...[or] communities that 
offered different programs, | couldn't get there to be able to get any benefit from them. 


[An organization] had a therapy group and | went to that; it was like an eight-week 
course...So | would sign up for things like that, but it’s very hard when you have that type 
of anxiety to even leave the house. So...it didn’t mean [I would] be able to take 
advantage of all eight weeks, because | may only be able to brave it two or three of 
those eight...times...So | missed a lot of the sessions. 


Internet-Delivered CBT 


Participants who had undergone iCBT described their experiences with the treatment process 
and what they felt were its benefits and limitations. During the treatment period, most had 
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access to their therapist via email and text messaging. Some participants also had in-person 
check-ins with their therapist. 


Treatment Process 


Most participants who had experienced iCBT reported that it improved the management of their 
mood and anxiety: 


It helped me a lot. It helped me to overcome a few of the things from my condition. 


Internet-based cognitive behavioural therapy is...the majority of the work is being done 
by the individual and managing anxiety and managing anxious thoughts really helps with 
a whole bunch of other mental health issues, like depression...1 would say...after 
learning how to manage my anxiety, | became...significantly less depressed because | 
wasnt spiralling down into these...negative-thinking loops. 


Interviewees mentioned that the earlier sessions in the course were simple, having been 
designed to help them better understand their condition, whereas the latter sessions were more 
complex and required intense reflection and thinking: 


So, it has information about the condition that is not just downloaded in a dump, but it’s 
bit by bit...depending on where you are in the treatment...certain things you need to 
know about that. And then a space on understanding that your thoughts...affect your 
behaviours. And then you go to a very concrete place where you start analyzing your 
thoughts with regard to the different situations that you're afraid [of]. 


Participants noted that each learning module was built on the previous one. For each module, 
they were required to complete homework assignments such as thought records during the 
week and send them to their therapist. 


Participants reported that the design, layout, and structure of the iCBT program were essential 
to keeping them engaged throughout the course of therapy. The usability and interactivity of the 
online platform were also important: 


| didn’t find that one [an iCBT program] particularly helpful...it was just not 
very...compelling or interactive; | didn’t really feel like there was a lot of structure to it. It 
felt really...overly self-motivated...you really had to...force yourself to do it, and | found 
that [made it] really challenging to stay motivated. The other one [another iCBT 
program] basically...has a much better...design and layout. They have...steps so you 
can see the entire program laid out before you, so you know what you're getting into. 
You know how much there is to do...it’s all planned out for you. 


Accountability and Support 


Many participants felt that motivation was key to completing an iCBT course. As many people 
with depression struggle with motivation, this is an important factor to consider for both 
practitioners and people considering pursuing iCBT. Interviewees who had experience with 
iCBT appreciated that the program provided access to a therapist who kept them accountable 
for their work at regular check-ins and provided support when needed: 
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The other thing that | found super helpful...[was] having...some sort of community 
support as well...however often you needed it... You are also able to text them 
[therapist/coach] whenever you have...an anxious moment. So, | found that...really 
helpful, because that helped me stay motivated, because | knew how much there was 
to do. 


Having weekly check-ins with a coach was also super helpful, because that helped 
me...to have somebody holding me accountable for what | was doing...It was just 
motivating to have somebody else out there who knew what you were going through 
and could help you through it and was...holding you accountable for it. 


Condition Severity 


Several participants with anxiety noted that the ability to do iCBT from home allowed them to 
access treatment that would otherwise have been inaccessible owing to the severity of their 
condition, which in some cases resulted in their being homebound. Some participants 
expressed that iCBT enabled them to manage their thoughts and readied them seek further 
treatment, if needed: 


It’s kind of like you have two broken legs...you can’t get to the doctor to go get casts on 
your legs; you need some help to be able to even get to the doctor's office. It’s the 
same thing...a program inside your home that would help you develop some crutches 
that [would allow you] to at least get out the door...to get additional help that’s 
needed...At least...develop skills to...leave and participate in something bigger. 


It [iCBT]...prompted me to...explore deeper and try to...figure out where the anxiety 
was coming from, like a root cause of it. And for that, | decided to go back to my 
therapist and...explore those reasons more. 


Control Over the Time and Location of Therapy 


Most participants who had experienced iCBT were satisfied with the amount of control over the 
time and location that iCBT offered. People who were in school or working reported that not 
needing to travel to appointments saved them time, making it easier for them to fit therapy into 
their busy schedules: 


The biggest plus point was the convenience and the flexibility of the timing. 


The convenience of deciding when to put it in the schedule...it just so nice to have the 
control. 


The first [benefit] is, when | was doing online therapy, what | really loved about that was 
the flexibility of it and the fact that | could do it from anywhere. So, | actually started it 
when | was...backpacking in Australia...and then | was able to continue when | got 
back to Canada, so the flexibility of that was great. 


| was not keen on taking leave from work and going and doing this [CBT] course 
because that would mean that | [would have to] take that leave from work, and | just 
wanted to do it online...at my own pace and convenience. 


| wanted to not miss work. | wanted to do it online at my own pace and convenience. 
There is no excuse...you can do appointments at any time, even at midnight. 
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Participants also found that the online iCBT platform reduced anxiety associated with 
scheduling appointments and travel time: 


Being at home and in a place where you’re comfortable [makes it] much easier to 
function, but as soon as you go out [in] public and you have to deal with things that 
you're not used to or things that change...noise levels or people or crowds, that is 
extremely difficult to deal with. 


For me...having to go to preplanned appointments...what if something [is] wrong on 
that day, [and] you're not in the mood to do it, then it can bring anxiety on. 


Control Over Pace and Length of Sessions 


With online access to iCBT educational materials, those pursuing a course of iCBT have access 
to their learning materials at all times. Participants reported that this encouraged them to 
regularly practise their skills and thus consolidate their learning and gain confidence in their new 
skills: 


As we learn more, as we practise, then we gain some confidence, we gain some 
confidence in that our symptoms become very predictable...You know, the internet is 
kind of nice because you can walk yourself 10 times [through] the same issue, and you 
can document that each time, or 10 times [through] the same crazy thinking...and it’s 
all documented...Then you can go back over that history, and you can start building the 
little muscles like going to the gym...a little bit at a time, build on your own confidence 
by practising. 


Participants reported that communicating with their therapist via email was helpful, in that it 
allowed them to plan what they wanted to say in advance. They also found the online interface 
useful, as it allowed them to save useful links, educational materials, and personal reflections on 
their computers for future reference: 


| get very flustered when | speak...so it was good just to...be able to...communicate in a 
way that | was able to get everything out, get the questions out that | needed. | like that 
she [therapist] was able to send links...that | could save onto my computer and | could 
just have...if | was having a bad day and | needed the techniques [to] kind of help to 
keep the anxiety down. 


There is a lot of thinking and analysis [in the iCBT] process, [one has to mentally] 
prepareto go through experiments. Therapists gives you feedback on [the] plan. There is 
[an] open line for questions. It is online through the system. Everything you write is kept 
there. 


Ease of Access 


Many noted that having online access to therapy was beneficial, given that when people are 
experiencing symptoms of depression or anxiety, they often want to be alone and also often 
spent time online. One participant noted that when he was experiencing symptoms of anxiety, 
he often browsed the internet. Then, when he had access to online therapy, he would choose to 
do therapy instead of randomly browse websites. Participants reported appreciating the ability to 
receive therapy and track their progress in the comfort and privacy of their homes with their 
personal devices: 
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The big benefit that | found was how accessible it was, so again because you can 
access it through your laptop, tablet, or phone. That was a big thing for me...Especially... 
the younger generation | feel would really benefit from that because, | mean, for me, 
especially when you're having anxiety or something, you tend to go and distract yourself 
by browsing different sites or going on YouTube. So rather than doing that, | would often 
use the app and just look over the material there to sort of cope with the anxiety or 
depression. 


Even though you didn't meet with your therapist face to face, it was really easy to 
communicate and...get...pretty detailed feedback, and [the process was] just pretty 
timely and responsive. And again, you sort of had a history of what you both talked 
about and the different exercises and recall for yourself. Since it was an app, you could 
actually see a graph and see a trend of how you were doing week by week. So, | found 
that was pretty handy, to kind of see how you'd improved in certain areas or also how 
certain different exercises helped improve your mood. 


| think check-in calls were...exactly what | needed...After a certain point, when | was 
certainly done with the [iCBT] program...! had these skills to...deal with my anxiety and 
my anxious thoughts, and...a way to...manage my thinking. 


Barriers to iCBT 


Although participants noted many benefits to iCBT, some barriers were also reported. 
Cost 


Although iCBT was less expensive than face-to-face CBT, the cost was considered a burden 
and potentially a barrier for people on limited income, students, or those who did not have 
insurance. 


There were some things that | could not work on. | had lost my job, and | had to drop it 
[therapy]. 


It was really pricy. | [student] believe | paid...around $250 for [each session]...basically, 

it still is a 12-week session, so | felt that was pretty expensive, especially if you consider 
the younger crowd who might be at school or college or...might not have that income to 

afford that app. 


Another important cost associated with iCBT is the cost of a computer, which some living ona 
low income reported being unable to afford: 


! could hardly afford a flip phone, let alone a computer. 


Though we think the whole world is wired, it’s not yet, and not everybody has a 
computer. 


There may be technology challenges because...all | have is a little phone to use the 
internet with, and my phone is so basic...[that] | can’t even load any kind of an app or 
anything on it because there's no room. The operating system takes up the whole 
phone, so it makes it really difficult for me to try and do things online. 
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They [people] may not be able to afford one [a computer]. So, access to a computer | 
think might be a challenge, it might not be, but it could be, and even if they could borrow 
a laptop or something from someone, | think understanding how to use it or something 
like that would be important. 


Computer and Internet Access 


Although free access to internet-accessible computers is available in public spaces such as 
libraries, some participants expressed concerns of privacy and confidentiality with regard to 
pursuing therapy in a public space on a shared computer: 


This isn't something I’d want to do over a public-access computer...Every time | talk to 
anybody about the lack of connectivity for those of us in the rural and remote areas, you 
know, the answer is always, “Well, there are public-access computers.” Well, I'm sorry, 
I'm not going to do psychotherapy over a public-access computer with people looking 
over my shoulder and people reading my stuff after | leave; that's not a suitable option. 


Participants in Northern Ontario also commented on internet speed as a barrier to pursuing 
iCBT: 


That's the biggest challenge with anything over the ‘net, because it’s fine in the cities, but 
when you get up here, up north, if you do have the means to have a computer and high- 
speed internet, the internet isn’t always available. For a lot of people, though, it’s just 
simply not accessible because of where they live. 


Computer Literacy 


Another important barrier to accessing iCBT is computer literacy. For those without the skills 
necessary to operate a computer or navigate the internet, iCBT is not an option. Participants 
noted that basic computer skills training would be needed for such individuals: 


Many have had a different past than | have. If they started very young before this whole 
internet age, they may not have the skill or ability to navigate a computer; they may not 
even own a computer. 


[There should be] a training manual of sorts...but sort of dumb it down as to the very 
basics of being able to use a tool, because some people may never have even had a 
computer. 


| know there [are] a lot of folks out there [who] don’t have the computer knowledge to do 
any online courses. | think that would overwhelm my partner because he’s not very 
computer sawy. 


Disabilities Affecting One’s Ability to Use a Computer 
Disability was also mentioned as a barrier to accessing iCBT: 
[For people] with disabilities, if you can’t speak, you can't type...[or] you can’t see, those 


are other challenges that come through with anything where it’s online or you have to 
read the contents [of educational materials]. 
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Language 


Participants who did not speak English fluently commented that language could be a barrier to 
receiving treatment. Some predominantly French-speaking participants indicated that they 
would not have been able to complete their course of iCBT, offered only in English, without the 
additional in-person assistance offered by their therapist. 


Limitations of iCBT 


Lack of Face-to-Face Interaction 


Some participants indicated that, based on their personality, they would have preferred face-to- 
face interaction with their therapist. Some mentioned that when they had been feeling 
particularly low, they wished they could have seen their therapist in person. One person 
mentioned that an in-person session would have allowed for an “emotional release.” Another 
expressed interest in using video-call applications such as Skype or FaceTime to interact with 
their therapist: 


| definitely felt like there could have been more person-to-person interaction, if not one- 
on-one, [then] in a group therapy setting. 


At first when | got the response back from my social worker, [it] made me...realize, 

“Oh, | kind of understand the way of my thinking, how it’s affecting me psychologically.” 
But then after a while [of] just receiving emails and never really even [hearing] the voice 
of the person [therapist], it just didn’t really have anything... It didn’t really help. 


Definitely the online...discussion forum helped, but...especially [at] my lower 
point...when | felt really isolated and alone, | think...being able to...maybe 
even...Skyping...the person running the course or something like that...! think would 
have been beneficial for me during those low points, or FaceTime or whatever software 
you want to use. 


Having that human connection [would be helpful], just being able to talk face to face 
with a professional, ask them questions, open [up] to them about what I’m 
experiencing. So, for me, I’m a very social person. | like to talk to people. | like being 
able to connect on that level. So just being able to talk to them, being able to open up, 
to cry if need be, just [to] have that emotional release [would be helpfull]. 


Given the format of iCBT, in which clients and therapists interact via email, participants 
sometimes had to wait for responses to questions about learning modules. Some noted that this 
wait could cause anxiety. As each iCBT learning module builds on the previous one, not getting 
Clarification in an appropriate amount of time can impede a person’s progress: 


Emailing and waiting to receive a response to a question would decrease effectiveness 
of the therapy. If you had misunderstood and perceived something incorrectly [during a 
module] and moved on [to the next module] as if you had perceived something 
incorrectly...it would decrease effectiveness of your learning. 


Every week she [therapist] would take a look at the homework and offer some 
suggestions. If | got stuck, usually it was the interpretation of something, [and] she 
would help me with that...People who are in trouble cannot always work [their] way 
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through [the content]...sometimes simply because we do not perceive the things the 
same way. 


Program Rigidity 


Some participants reported that the iCBT program appears to be prescriptive and rigid. Unlike 
in-person CBT, which allows clients the opportunity to explore issues as they surface, iCBT 
follows a more rigid course that does not allow for issues to be explored as they arise. One 
person reported feeling they had to “stick to the regimen of the computer program.” 


A few interviewees compared iCBT with a self-help book, in that neither allows for flexibility in 
terms of therapeutic content (what topics are covered) or personal preference (e.g., for face-to- 
face interaction with a therapist). One person mentioned that waiting for a text or email 
response to a question did not meet their expectation of receiving immediate responses: 


Doing it online would be just the same as picking up the book and reading it. And, you 
know, | guess that’s my thing with it...I think the contact part is important for somebody 
like me. Maybe not everybody, but [for] somebody like me, it’s important. 


The whole program made me feel like | could just buy a self-help book and just do it 
that way. So, for me, being totally internet based and never being in contact [with the 
therapist], you know, face to face or anything like that, | think that was the main reason 
it did not help. 


| stopped around halfway [through the program]...because...the person [therapist] is 
actually really not helping me. So that’s why I’m actually doing it [therapy] in person at 
the moment. 


Condition Severity 


Most participants indicated that because undergoing CBT is an emotionally challenging 
experience, internet-based therapy may not be the best option for people with severe 
depression. Participants mentioned that when they are extremely depressed, they often lack the 
motivation to undergo iCBT, as this therapy requires more motivation than others: 


And | feel like, definitely, my case was a bit more...! guess a bit of a complex 
case...So, | definitely feel...1 guess [if] the province were to move forward with an 
OHIP-covered CBT program, there should be an [option for] individual or group 
therapy...[and a] face-to-face component for more complex cases. It shouldn't be a 
one-size-fits-all solution. 


Participants also felt that people with certain anxieties related to privacy implications from the 
government or internet communication may not be able to benefit from iCBT: 


Given the online platform and people recording their thoughts online, this can cause 
anxiety for people who are not comfortable sharing their inner thinking over the 
internet. 


Some people are not confident with the government. Folks, too, might not be from 
Canada [and may] feel like they can’t trust the government to open up and be honest 
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about things that they don’t want to be discriminated against. That could be a barrier for 
some people, too. 


As with in-person CBT, interviewees noted that the length of a course of iCBT treatment is 
important, stating that those whose condition is more severe may benefit from a longer course 
of treatment: 


Other people get [understand] it the first time, but it may be their first episode of anxiety 
or depression for...but for somebody who has a serious illness, whether it’s bipolar or a 
serious anxiety from a phobia or...PTSD [post-traumatic stress disorder]...Like in the 
early 90s and through the ’90s...if somebody needed longer-term care, they got that, 
and they weren't just kicked out after 10 or 12 sessions. 


Comprehension of Information 


A few participants noted frustration at their inability to comprehend long emails and sessions as 
therapy became more complex. They explained that their condition impacted their cognitive 
functioning and that in-person sessions would have allowed them to interrupt their therapist and 
ask questions when they needed clarification: 


Just getting...emails explaining the way of thinking, it didn't really...do anything... 
compared to...[a] personal one [in-person therapy]. It [in-person therapy] helps a lot 
more...because.../’m there. | can even ask if | don’t understand something, what she’s 
explaining; then | can...interrupt and ask what that really means, because having this 
condition, my comprehension is really, | would say, degraded, and | think having 
human contact actually helps me more to really understand the concept of the whole 
therapy. 


It was user friendly, but the amount of information was getting more and more in-depth, 
which [it] is supposed to be, | assume, and again it’s based on how much | can 
comprehend...It was just too much as well for me. It was...a lot more complex and 
halfway done already, and | just felt lost, basically, in all that information. 


Follow-Up Support 


As mentioned, some participants reported that, following therapy, increased stress and changes 
in routine could have a negative impact on their thought processes that could result in 
succumbing to their old ways of thinking. As with in-person CBT, participants mentioned that 
people could likely benefit from some type of follow-up, coaching, or peer support group 
following the completion of iCBT. Further, they felt that people could benefit from this process as 
both the person receiving the follow-up support and as a person providing such support: 


For me personally, | found that | didn’t have a whole lot of...support outside. And...1 
guess since we're talking about...online cognitive therapy, | think that stuff is really 
good...! think a key to it is follow-up. It’s almost like you learn these things, and then 
you need...some coaching or mentoring on how to put the theory into action in some 
cases. 


| also think that there is a need for...having...a physical presence and physical 
engagement, some sort of...supportive community around it as well. 
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You know the best way to keep it going is to teach it to others. So, if you have an online 
[therapy program] that you’re creating, | would create some kind of community [to go 
along with it]. People like this have had nobody, and if they start learning to have 
confidence, they’re going to learn from each other, and...in teaching other people, they 
reinforce their own learning, and they no longer become the victim; they become, you 
know, like their life has some meaning. 


Technical Difficulties 


Participants noted that their experience of therapy could be disrupted owing to technical 
difficulties with the online iCBT program: 


There [were] a few weeks when there was some bug or something where | had 
problems accessing the app, so even though | was sort of able to go and log in through 
their site and was able to do the weekly stuff, there [were] some technical issues [that] 
were annoying. 


Discussion 


People with depression or anxiety shared their experiences of the burden their condition has on 
their daily life, self-esteem, relationships, work, and finances, as well as their experiences with a 
number of treatment options. 


Interviewees indicated that medication and in-person therapy are currently available treatment 
options, with most reporting being able to access medication. However, many found that 
medications often had undesirable side effects, and many felt that medication was not a long- 
term solution for their condition. People with experience of in-person CBT indicated that it had 
therapeutic benefits, but that barriers such as cost, and access exist. Several limitations to in- 
person CBT were reported, including the time for therapy and the time for travel to and from 
appointments, the number of sessions in a course of treatment, symptom severity sometimes 
making it difficult or impossible to attend therapy, and the lack of follow-up support. 


Most interviewees with experience of iCBT reported that it had a positive impact on their 
condition. Many reported perceiving that their iCBT therapist had provided them with an 
appropriate degree of accountability and support. Reported benefits of iCBT versus in-person 
CBT include increased flexibility in and control over the time, pace, and location of therapy. 
People who were homebound owing to the severity of their condition reported that iCBT 
provided them an option for therapy that otherwise would not be available to them, and that it 
helped them develop the skills needed to leave home for further treatment if needed. The online 
platform of iCBT allowed easier access and reference to learning materials on personal devices. 
Reported barriers to accessing iCBT included the cost of therapy and the need for a computer, 
internet access, computer literacy, and the ability to comprehend complex written information. 
Language was also reported as a barrier for the people we interviewed as they reported that 
iCBT was offered only in English. Reported limitations of iCBT included the lack of in-person 
interaction, the rigidity of the program, symptom severity affecting people’s motivation to pursue 
therapy, the number of sessions in a course of treatment, the lack of follow-up support, and the 
potential for technical difficulties with the online platform. 
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Conclusions 


People with depression or anxiety with whom we spoke viewed both in-person CBT and iCBT 
as effective treatment options for adults with depression or anxiety. Internet-delivered CBT may 
be especially beneficial for those with whose symptoms may prevent them from leaving home. 
Internet-delivered CBT may also be beneficial for those with busy schedules who would find it 
difficult to arrange travel time to and from appointments, and for those for whom the cost of 
travel to and from appointments would be a barrier. However, iCBT is associated with important 
barriers and limitations, including the need for a computer, internet access, computer literacy, 
and the ability to comprehend complex written information. 


For both in-person CBT and iCBT, the cost of treatment, the number of sessions in a course of 
treatment, and the lack of follow-up support were reported as substantial drawbacks. 


In-person CBT may be preferred by people who feel they would benefit from in-person 
interaction and the opportunity to deal with issues as they arise, whereas iCBT may be 
preferred by those wanting to pursue therapy from the comfort of home and who are 
comfortable adhering to a more structured course of therapy. 


For people with mild to moderate depression or anxiety, iCBT provided control over the time, 
pace, and location of therapy. Despite some perceived limitations, iCBT was felt to improve 
access for those people who could not otherwise access treatment due to costs, time, or nature 
of their health condition. 
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CONCLUSIONS OF THE HEALTH TECHNOLOGY ASSESSMENT 


Compared to waiting list, guided iCBT significantly improves symptoms of mild to moderate 
major depression (GRADE moderate), generalized anxiety disorder (GRADE low), panic 
disorder (GRADE low), and soical phobia (GRADE low). Compared to group or individual face- 
to-face CBT, guided iCBT did not significantly improve symptoms of panic disorder (GRADE 
very low). 


In various cost-utility analyses, we found that over a short-term time horizon guided iCBT 
represents good value for money compared with other available options such as unguided 
iCBT, face-to-face CBT, and usual care. Guided iCBT could be offered as an initial step for the 
short-term treatment of eligible adults with mild to moderate major depression or anxiety 
disorders. Assuming a 3% increase in access per year over the next 5 years, the annual net 
budget impact of publicly funding guided iCBT in Ontario would be between $10 million and 
$40 million for the treatment of mild to moderate major depression, and between $16 million 
and $65 million for the treatment of anxiety disorders. 


People with depression or anxiety with whom we spoke viewed iCBT as an effective treatment 
option. Internet-delivered CBT may be preferred over in-person CBT by those wanting to pursue 
therapy from home, those with busy schedules, those who are comfortable adhering to a more 
structured course of therapy, and those with severe anxiety preventing them from leaving home. 
Despite some perceived limitations, iCBT was felt to improve access to treatment for people 
who may not otherwise be able to access treatment owing to cost, time, or the severity of their 
health condition. 
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ABBREVIATIONS 
CADTH Canadian Agency for Drugs and Technologies in Health 
Cl confidence interval 
DSM Diagnostic and Statistical Manual of Mental Disorders 
GAD generalized anxiety disorder 
GP general practitioner 
GRADE Grading of Recommendations Assessment, Development, and 
Evaluation 
IAPT Improving Access for Psychological Therapies 
iCBT internet-delivered cognitive behavioural therapy 
ICER incremental cost-effectiveness ratio 
MDE major depressive episode 
MSW Master’s of Social Work 
OCD obsessive-compulsive disorder 
OR odds ratio 
PTSD post-traumatic stress disorder 
QALY quality-adjusted life-year 
RR relative risk 
SD standard deviation 
SMD standardized mean difference 
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Budget impact analysis 


Cost-effective 


Cost-utility analysis 


Discounting 
Dominant 


Economically attractive 
Incremental cost- 
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A technique to estimate the financial impact of a planned action 
over a specified time period by calculating the costs and savings of 
different options. 

Good value for money. The overall benefit of the technique or 
intervention justifies the cost. 

A type of analysis that estimates the value for money of an 
examined intervention versus control treatment (e.g., usual care). 
The result is expressed as a dollar amount per “quality-adjusted 
life-year” or QALY. 

A method that considers that costs and health benefits are worth 
more today than in the future. 

An intervention is dominant when it is both more effective and less 
costly than its alternative comparator(s). 

See Cost-effective. 

Determines “a unit of benefit” for an intervention by dividing the 


effectiveness ratio (ICER) incremental cost by the incremental effectiveness. The incremental 


Markov model 


Ministry of Health and 
Long-Term Care 
perspective 


Net budget impact 


Quality-adjusted life-year 


(QALY) 


Reference case 


Societal perspective 


cost is the difference between the mean costs of the treatment 
under study and an alternative treatment. The incremental 
effectiveness is the additional unit of health effect gained (e.g., life- 
years or QALYs). 

A type of modelling that measures the health state of a patient over 
the course of treatment. A patient may stay in one health state or 
move from one health state to another, depending on the effect of 
the treatment and the progression of the disease. 

The Ministry of Health and Long-Term care perspective includes 
treatment costs (e.g., drug, administration, monitoring, and health 
service resource use, such as hospital stays and general 
practitioner visits) and costs associated with managing adverse 
events caused by treatment. This perspective does not include 
patients’ out-of-pocket costs relating to obtaining care (e.g., 
transportation) or loss of productivity costs (e.g., absenteeism, 
presenteeism) 

In a budget impact analysis, the net budget impact is the difference 
in cost between a future scenario (new treatment) as compared 
with the current scenario (usual care). 

A measurement that considers both the number of years gained by 
a patient from a procedure and the quality of those extra years 
(considering such factors as ability to function and freedom from 
pain). The QALY is commonly used as an outcome measure in 
cost-utility analyses. 

A set of recommended methods used for all evaluations that 
promote uniformity and transparency and that enable the 
comparison of results for different technologies and different 
decisions. 

The societal perspective considers the full effect on society of a 
condition, including all costs, regardless of who pays, and all 
benefits, regardless of who receives the benefits. 
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Time horizon Costs and outcomes are examined within a chosen time frame. In 
an economic evaluation, this time frame is referred to as the time 
horizon. 
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APPENDICES 


Appendix 1: Literature Search Strategies 
Clinical Evidence Search 
Search date: February 15, 2018 


Databases searched: All Ovid MEDLINE, Embase, Cochrane Database of Systematic 
Reviews, CRD Health Technology Assessment Database, NHS Economic Evaluation Database, 
PsycINFO, CINAHL 


Ovid Database: EBM Reviews - Cochrane Database of Systematic Reviews <2005 to February 
14, 2018>, EBM Reviews - Health Technology Assessment <4th Quarter 2016>, EBM Reviews 
- NHS Economic Evaluation Database <ist Quarter 2016>, Embase <1980 to 2018 Week 07>, 
All Ovid MEDLINE(R) <1946 to Present>, PsycINFO <1806 to February Week 1 2018> 


Search Strategy 

Depression/ (431591) 

Depressive Disorder/ (128784) 

Depressive Disorder, Major/ (30842) 

Depressive Disorder, Treatment-Resistant/ (1071) 

(depress* or MDD).ti,ab,kf. (1169437) 

Anxiety/ (288598) 

Anxiety Disorders/ (55241) 

Agoraphobia/ (10906) 

Anxiety, Separation/ (4124) 

10 Panic Disorder/ (26098) 

11 exp Phobic Disorders/ (37106) 

12 Mutism/ (3684) 

13 (anxiet* or anxious* or panic* or phobi* or agoraphobi* or GAD or mute or mutism).ti,ab,kf. 
(636431) 

14 or/1-13 (1733300) 

15 Cognitive Therapy/ (76265) 

16 (((cognitive or behavio*) adj2 (therap* or psychotherap*)) or cognitive behavio* or cognition 
therap* or CBT*).ti,ab,kf. (142208) 

17 15 0r 16 (171612) 

18 Internet/ (185887) 

19 Therapy, Computer-Assisted/ (10344) 

20 Computer-Assisted Instruction/ (90169) 

21 Mobile Applications/ (7010) 

22 Telemedicine/ (38968) 

23 Remote Consultation/ (11963) 

24 (internet* or Beacon or app or apps or (mobile adj2 application*) or smartphone* or smart 
phone* or mobile based or e mail* or email* or electronic mail* or "Information and 
communication technology" or "Information and communication technologies" or emedicine or e 
medicine or ehealth* or e health* or emental health* or e mental health* or etherap* or e therap* 
or epsychiatr* or e psychiatr* or epsychol* or e psychol* or telemedicine or tele medicine or 
telehealth* or tele health* or telemental health* or tele mental health* or telecare or tele care or 


OONDABRWN — 
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teletherap* or tele therap* or telepsychiatr* or tele psychiatr* or telepsychol* or tele 
psychol").ti,ab,kf. (279138) 
25 ((technolog* or computer* or digital* or webbased or web based or webdeliver* or web 
deliver* or online) adj6 (therap* or psychotherap* or CBT or intervention or treatment* or 
deliver* or technique™ or training)).ti,ab,kf. (162924) 
26 or/18-25 (635529) 
27 14 and 17 and 26 (5032) 
28 (eCBT* or cCBT* or iCBT*).ti,ab,kf. (1908) 
29 14 and 28 (1194) 
30 (MoodGym or Big White Wall or Beating the Blues or Fear Fighter or E compass or 
Ecompass or Deprexis or Moodkit or Living Life to the Full).ti,ab,kf. (314) 
31 27 or 29 or 30 (5318) 
32 Meta Analysis.pt. (84720) 
33 Meta-Analysis/ or Meta-Analysis as Topic/ or exp Technology Assessment, Biomedical/ 
(286547) 
34 (((systematic* or methodologic’) adj3 (review* or overview*)) or pooled analysis or 
published studies or published literature or hand search* or handsearch* or medline or pubmed 
or embase or cochrane or cinahl or data synthes* or data extraction* or HTA or HTAs or 
(technolog* adj (assessment* or overview* or appraisal*))).ti,ab. (645525) 
5 (meta analy* or metaanaly* or health technolog* assess*).mp. (437423) 
36 or/32-35 (881233) 
37 31 and 36 (680) 
38 Case Reports/ or Congresses.pt. (1930375) 
39 37 not 38 (680) 
40 limit 39 to english language [Limit not valid in CDSR; records were retained] (655) 
41 limit 40 to yr="2000 -Current" (647) 
42 41 use ppez,cleed (204) 
43 limit 31 to english language [Limit not valid in CDSR; records were retained] (5120) 
44 limit 43 to yr="2000 -Current" (501 1) 
45 44 use coch,clhta (15) 
46 depression/ (431591) 
47 major depression/ (162267) 
48 (depress* or MDD).tw,kw. (1186504) 
49 anxiety/ (288598) 
50 anxiety disorder/ (104805) 
51 generalized anxiety disorder/ (11358) 
52 panic/ (25444) 
53 phobia/ (26970) 
54 agoraphobia/ (10906) 
55 claustrophobia/ (11329) 
56 social phobia/ (14558) 
57 separation anxiety/ (7268) 
58 anxiety neurosis/ (25355) 
59 "mixed anxiety and depression"/ (538) 
60 selective mutism/ (1622) 
61 (anxiet* or anxious* or panic* or phobi* or agoraphobi* or GAD or mute or mutism).tw,kw. 
(650368) 
62 or/46-61 (1750576) 
63 cognitive therapy/ (76265) 
64 cognitive behavioral therapy/ (25567) 
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65 (((cognitive or behavio*) adj2 (therap* or psychotherap*)) or cognitive behavio* or cognition 
therap* or CBT*).tw,kw,dv. (150690) 

66 or/63-65 (180094) 

67 Internet/ (185887) 

68 computer assisted therapy/ (11486) 

69 computer program/ (279888) 

70 e-mail/ (18195) 

71 mobile application/ (7881) 

72 computer/ (121808) 

73 personal computer/ (15611) 

74  software/ (120953) 

75 telecommunication/ (28846) 

76 teleconference/ (5799) 

77 teleconsultation/ (12436) 

78 telemedicine/ (38968) 

79 telepsychiatry/ (453) 

80 teletherapy/ (7848) 

81. (internet* or Beacon or app or apps or (mobile adj2 application*) or smartphone* or smart 
phone* or mobile based or e mail* or email* or electronic mail* or "Information and 
communication technology" or "Information and communication technologies" or emedicine or e 
medicine or ehealth* or e health* or emental health* or e mental health* or etherap* or e therap* 
or epsychiatr* or e psychiatr* or epsychol* or e psychol* or telemedicine or tele medicine or 
telehealth* or tele health* or telemental health* or tele mental health* or telecare or tele care or 
teletherap* or tele therap* or telepsychiatr* or tele psychiatr* or telepsychol* or tele 
psychol*).tw,kw,dv. (289515) 

82 ((technolog* or computer* or digital* or webbased or web based or webdeliver* or web 
deliver* or online) adj6 (therap* or psychotherap* or CBT or intervention* or treatment* or 
deliver* or technique™ or training)).tw,kw,dv. (167557) 

83 or/67-82 (977112) 

84 62 and 66 and 83 (5851) 

85 (eCBT* or cCBT* or iCBT*).tw,kw,dv. (1960) 

86 62 and 85 (1239) 

87 (MoodGym or Big White Wall or Beating the Blues or Fear Fighter or E compass or 
Ecompass or Deprexis or Moodkit or Living Life to the Full).tw,kw. (326) 

88 84 or 86 or 87 (6148) 

89 Meta Analysis/ or "Meta Analysis (Topic)"/ or Biomedical Technology Assessment/ 
(280947) 

90 (((systematic* or methodologic’) adj3 (review* or overview*)) or pooled analysis or 
published studies or published literature or hand search* or handsearch* or medline or pubmed 
or embase or cochrane or cinahl or data synthes* or data extraction* or HTA or HTAs or 
(technolog* adj (assessment* or overview* or appraisal*))).ti,ab. (645525) 

91 (meta analy* or metaanaly* or health technolog* assess*).mp. (437423) 

92 or/89-91 (880146) 

93 88 and 92 (1113) 

94 Case Report/ or conference abstract.pt. (6872119) 

95 93 not 94 (1079) 

96 limit 95 to english language [Limit not valid in CDSR; records were retained] (1051) 

97 limit 96 to yr="2000 -Current" (1011) 

98 97 use emez (302) 

99 "depression (emotion)"/ (123407) 

100 major depression/ (162267) 
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101 recurrent depression/ (749) 

102 treatment resistant depression/ (4640) 

103 (depress* or MDD).ti,ab,id. (1170012) 

104 anxiety/ (288598) 

105 anxiety disorders/ (55241) 

106 generalized anxiety disorder/ (11358) 

107 panic disorder/ (26098) 

108 panic attack/ (28367) 

109 social anxiety/ (75650) 

110 exp phobias/ (22840) 

111 separation anxiety disorder/ (5612) 

112 Elective Mutism/ (1485) 

113 (anxiet* or panic* or phobi* or agoraphobi* or GAD or mute or mutism).ti,ab,id. (617046) 
114 or/99-113 (1636314) 

115 cognitive behavior therapy/ (42521) 

116 cognitive therapy/ (76265) 

117 (((cognitive or behavio*) adj2 (therap* or psychotherap”*)) or cognitive behavio* or 
cognition therap* or CBT*).ti,ab,id. (145381) 

118 or/115-117 (176194) 

119 Internet/ (185887) 

120 Computer Assisted Therapy/ (11486) 

121 Computer Mediated Communication/ (5218) 

122 computer assisted instruction/ (90169) 

123 computer software/ (105562) 

124 computer applications/ (11089) 

125 mobile devices/ (1843) 

126 electronic communication/ (3031) 

127 human computer interaction/ (15088) 

128 Information Technology/ (16892) 

129 electronic learning/ (660) 

130 online therapy/ (2362) 

131 Telemedicine/ (38968) 

132 (internet* or Beacon or app or apps or (mobile adj2 application*) or smartphone* or smart 
phone* or mobile based or e mail* or email* or electronic mail* or "Information and 
communication technology" or "Information and communication technologies" or emedicine or e 
medicine or ehealth* or e health* or emental health* or e mental health* or etherap* or e therap* 
or epsychiatr* or e psychiatr* or epsychol* or e psychol* or telemedicine or tele medicine or 
telehealth* or tele health* or telemental health* or tele mental health* or telecare or tele care or 
teletherap* or tele therap* or telepsychiatr* or tele psychiatr* or telepsychol* or tele 
psychol*).ti,ab,id. (278495) 

133 ((technolog* or computer* or digital* or webbased or web based or webdeliver* or web 
deliver* or online) adj6 (therap* or psychotherap* or CBT or intervention or treatment* or 
deliver* or technique™ or training)).ti,ab,id. (163192) 

134 or/119-133 (747179) 

135 114 and 118 and 134 (4972) 

136 (eCBT* or cCBT* or iCBT").ti,ab,id. (1899) 

137 114 and 136 (1176) 

138 (MoodGym or Big White Wall or Beating the Blues or Fear Fighter or E compass or 
Ecompass or Deprexis or Moodkit or Living Life to the Full).ti,ab,id. (815) 

139 135 or 137 or 138 (5256) 
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140 (((systematic* or methodologic’) adj3 (review* or overview*)) or pooled analysis or 


February 2019 


published studies or published literature or hand search* or handsearch* or medline or pubmed 


or embase or cochrane or cinahl or data synthes* or data extraction* or HTA or HTAs or 


(technolog* adj (assessment* or overview* or appraisal*))).ti,ab. (645525) 


141 
142 
143 
144 
145 
146 
147 
148 
149 
150 
151 
152 
153 


(meta analy* or metaanaly* or health technolog* assess*).mp. (4387423) 
(systematic review or meta analysis).md. (33994) 

or/140-142 (866376) 

139 and 143 (673) 

case report/ (4089061) 

144 not 145 (671) 


limit 146 to english language [Limit not valid in CDSR; records were retained] (644) 


limit 147 to yr="2000 -Current" (636) 
148 use psyh (126) 

42 or 45 or 98 or 149 (647) 

150 use ppez (204) 

150 use coch (7) 

150 use clhta (8) 


154 150 use cleed (0) 

155 150 use emez (302) 

156 150 use psyh (126) 

CINAHL 

# Query Results 

S1_ | (MH "Depression") 81,242 

S2_ | depress* OR MDD 135,715 

$3 | (MH "Anxiety") 30,406 

S4 | (MH "Anxiety Disorders") 8,352 

S5_ | (MH "Generalized Anxiety Disorder") 289 

S6_ | (MH "Panic Disorder") 1,841 

S7_ | (MH "Phobic Disorders+") 4,368 

S8 | (MH "Separation Anxiety") 546 

S9_ | (MH "Mutism") 223 
anxiet* OR anxious* OR panic* OR phobi* OR agoraphobi* OR GAD OR mute 

$10 | OR mutism 75,834 

$11) S1 OR S2 OR S3 OR S4 OR S5 OR S6 OR S7 OR S8 OR S9 OR S10 178,201 

$12 | (MH "Cognitive Therapy") 15,055 
((cognitive OR behavio*) N2 (therap* OR psychotherap*)) OR cognitive 

$13 | behavio* OR cognition therap* OR CBT* 39,189 
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$14) S120R S13 39,189 
$15 | (MH "Internet") 38,850 
$16 | (MH "World Wide Web") 63,831 
$17 | (MH "World Wide Web Applications") 4,682 
$18 | (MH "Therapy, Computer Assisted") 4,731 
$19 | (MH "Mobile Applications") 2,897 
$20 | (MH "Telehealth") 5,018 
$21 | (MH "Telemedicine") 7,577 
$22 | (MH "Telepsychiatry") 231 
$23 | (MH "Remote Consultation") 1,481 


internet* OR Beacon OR app OR apps OR (mobile N2 application*) OR 
smartphone* OR smart phone* OR mobile based OR e mail* OR email* OR 
electronic mail* OR "Information and communication technology" OR 
"Information and communication technologies" OR emedicine OR e medicine 
OR ehealth* OR e health* OR emental health* OR e mental health* OR 
etherap* OR e therap* OR epsychiatr* OR e psychiatr* OR epsychol* OR e 


$24 | psychol* 72,532 
telemedicine OR tele medicine OR telehealth* OR tele health* OR telemental 
health* OR tele mental health* OR telecare OR tele care OR teletherap* OR 
tele therap* OR telepsychiatr* OR tele psychiatr* OR telepsychol* OR tele 
$25 | psychol* 14,496 
((technolog* OR computer* OR digital* OR webbased OR web based OR 
webdeliver* OR web deliver* OR online) N6 (therap* OR psychotherap* OR 
$26 | CBT OR intervention* OR treatment* OR deliver* OR technique* OR training)) 24,487 
S15 OR S16 OR S17 OR S18 OR S19 OR S20 OR S21 OR S22 OR S23 OR 
S27 | S24 OR S25 OR S26 165,602 
$28 | S11 AND S14 AND S27 997 
$29 | eCBT* or iCBT* or cCBT* 205 
$30 | S11 AND S29 148 
MoodGym OR Big White Wall OR Beating the Blues OR Fear Fighter OR E 
S31 | compass OR Ecompass OR Deprexis OR Moodkit OR Living Life to the Full 144 
$32 | S28 OR S30 OR S31 1,123 
$33 | (MH "Meta Analysis") 31,096 
$34 | (PT "Meta Analysis") or (PT "Systematic Review") 74,036 
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((systematic* or methodologic*) N3 (review* or overview*)) or pooled analysis or 
published studies or published literature or hand search* or handsearch* or 
medline or pubmed or embase or cochrane or cinahl or data synthes* or data 
extraction* or HTA or HTAs or (technolog* N1 (assessment* or overview* or 
$35 | appraisal*)) 148,301 
$36 | S33 OR S34 OR S35 159,578 
$37 | S32 AND S36 124 
$38 | (MH "Case Studies") OR (MH "Congresses and Conferences") 89,938 
$39 | S37 NOT S38 124 
$37 NOT S38 
$40 | Limiters - Published Date: 20000101-; English Language 123 


Economic Evidence Search 
Search date: February 21, 2018 


Databases searched: All Ovid MEDLINE, Embase, Cochrane Database of Systematic 
Reviews, Cochrane Central Register of Controlled Trials, CRD Health Technology Assessment 
Database, NHS Economic Evaluation Database, PsycINFO, CINAHL 


Ovid Database: EBM Reviews - Cochrane Central Register of Controlled Trials <January 
2018>, EBM Reviews - Cochrane Database of Systematic Reviews <2005 to February 14, 
2018>, EBM Reviews - Health Technology Assessment <4th Quarter 2016>, EBM Reviews - 
NHS Economic Evaluation Database <1st Quarter 2016>, Embase <1980 to 2018 Week 08>, 
All Ovid MEDLINE(R) <1946 to Present>, PsycINFO <1806 to February Week 2 2018> 


Search Strategy 

Depression/ (439538) 

Depressive Disorder/ (133466) 

Depressive Disorder, Major/ (33905) 

Depressive Disorder, Treatment-Resistant/ (1225) 
(depress* or MDD).ti,ab,kf. (1218104) 

Anxiety/ (295392) 

Anxiety Disorders/ (57570) 

Agoraphobia/ (11298) 

Anxiety, Separation/ (4190) 

10 Panic Disorder/ (26885) 

11 exp Phobic Disorders/ (38122) 

12 Mutism/ (3694) 

13 (anxiet* or anxious* or panic* or phobi* or agoraphobi* or GAD or mute or mutism).ti,ab,kf. 
(664501) 

14 or/1-13 (1800358) 

15 Cognitive Therapy/ (82916) 


OONOAARWN— 
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16 (((cognitive or behavio*) adj2 (therap* or psychotherap*)) or cognitive behavio* or cognition 
therap* or CBT*).ti,ab,kf. (155741) 

17 15 or 16 (186922) 

18 Internet/ (189187) 

19 Therapy, Computer-Assisted/ (11416) 

20 Computer-Assisted Instruction/ (91414) 

21 Mobile Applications/ (7298) 

22 Telemedicine/ (40471) 

23 Remote Consultation/ (12309) 

24 (internet* or Beacon or app or apps or (mobile adj2 application*) or smartphone* or smart 
phone* or mobile based or e mail* or email* or electronic mail* or "Information and 
communication technology" or "Information and communication technologies" or emedicine or e 
medicine or ehealth* or e health* or emental health* or e mental health* or etherap* or e therap* 
or epsychiatr* or e psychiatr* or epsychol* or e psychol* or telemedicine or tele medicine or 
telehealth* or tele health* or telemental health* or tele mental health* or telecare or tele care or 
teletherap* or tele therap* or telepsychiatr* or tele psychiatr* or telepsychol* or tele 
psychol*).ti,ab,kf. (288011) 

25 ((technolog* or computer* or digital* or webbased or web based or webdeliver* or web 
deliver* or online) adj6 (therap* or psychotherap* or CBT or intervention* or treatment* or 
deliver* or technique™ or training)).ti,ab,kf. (172863) 

26 or/18-25 (655134) 

27 14 and 17 and 26 (6066) 

28 (eCBT* or iCBT* or cCBT").ti,ab,kf. (2272) 

29 14 and 28 (1459) 

30 (MoodGym or Big White Wall or Beating the Blues or Fear Fighter or E compass or 
Ecompass or Deprexis or Moodkit or Living Life to the Full).ti,ab,kf. (393) 

31 27 or 29 or 30 (6424) 

32 economics/ (276061) 

33 economics, medical/ or economics, pharmaceutical/ or exp economics, hospital/ or 
economics, nursing/ or economics, dental/ (795242) 

34 economics.fs. (400763) 

35 (econom* or price or prices or pricing or priced or discount* or expenditure* or budget* or 
pharmacoeconomic* or pharmaco-economic’).ti,ab,kf. (933453) 

36 exp "costs and cost analysis"/ (572001) 

37 (cost or costs or costing or costly).ti. (252770) 

38 cost effective*.ti,ab,kf. (294433) 

39 (cost* adj2 (util* or efficacy* or benefit* or minimi* or analy* or saving* or estimate* or 
allocation or control or sharing or instrument* or technolog*)).ab,kf. (199167) 

40 models, economic/ (11111) 

41 markov chains/ or monte carlo method/ (72649) 

42 (decision adj1 (tree* or analy* or model*)).ti,ab,kf. (89787) 

43 (markov or markow or monte carlo).ti,ab,kf. (120475) 

44 quality-adjusted life years/ (84469) 

45 (QOLY or QOLYs or HRQOL or HRQOLs or QALY or QALYs or QALE or QALEs).ti,ab,kf. 
(64871) 

46 ((adjusted adj2 (quality or life)) or (willing* adj2 pay) or sensitivity analys*s).ti,ab,kf. 
(104322) 

47 or/32-46 (2532592) 

48 31 and 47 (1027) 

49 Case Reports/ or Comment.pt. or Editorial.pt. or Letter.pt. or Congresses.pt. (4894150) 
50 48 not 49 (1016) 
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51 limit 50 to english language [Limit not valid in CDSR; records were retained] (993) 

52. limit 51 to yr="2000 -Current" (977) 

53 52 use ppez,coch,cctr,clhta (416) 

54 limit 31 to english language [Limit not valid in CDSR; records were retained] (6153) 

55 limit 54 to yr="2000 -Current" (6037) 

56 55 use cleed (17) 

57 53 or 56 (433) 

58 depression/ (439538) 

59 major depression/ (162568) 

60 (depress* or MDD).tw,kw. (1236728) 

61 anxiety/ (295392) 

62 anxiety disorder/ (107266) 

63 generalized anxiety disorder/ (11394) 

64 panic/ (25750) 

65 phobia/ (27948) 

66 agoraphobia/ (11298) 

67 claustrophobia/ (12286) 

68 social phobia/ (15545) 

69 separation anxiety/ (7339) 

70 anxiety neurosis/ (25407) 

71 "mixed anxiety and depression"/ (541) 

72 selective mutism/ (1629) 

73 (anxiet* or anxious* or panic* or phobi* or agoraphobi* or GAD or mute or mutism).tw,kw. 
(679930) 

74 or/58-73 (1819572) 

75 cognitive therapy/ (82916) 

76 cognitive behavioral therapy/ (25701) 

77 (((cognitive or behavio*) adj2 (therap* or psychotherap”*)) or cognitive behavio* or cognition 
therap* or CBT*).tw,kw,dv. (165848) 

78 or/75-77 (196948) 

79 Internet/ (189187) 

80 computer assisted therapy/ (12560) 

81 computer program/ (280859) 

82 e-mail/ (18512) 

83 mobile application/ (7961) 

84 computer/ (121857) 

85 personal computer/ (15849) 

86 software/ (122285) 

87 telecommunication/ (28938) 

88 teleconference/ (5880) 

89 teleconsultation/ (12782) 

90 telemedicine/ (40471) 

91 telepsychiatry/ (456) 

92 teletherapy/ (7959) 

93 (internet* or Beacon or app or apps or (mobile adj2 application*) or smartphone* or smart 
phone* or mobile based or e mail* or email* or electronic mail* or "Information and 
communication technology" or "Information and communication technologies" or emedicine or e 
medicine or ehealth* or e health* or emental health* or e mental health* or etherap* or e therap* 
or epsychiatr* or e psychiatr* or epsychol* or e psychol* or telemedicine or tele medicine or 
telehealth* or tele health* or telemental health* or tele mental health* or telecare or tele care or 
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teletherap* or tele therap* or telepsychiatr* or tele psychiatr* or telepsychol* or tele 
psychol*).tw,kw,dv. (298897) 

94 ((technolog* or computer* or digital* or webbased or web based or webdeliver* or web 
deliver* or online) adj6 (therap* or psychotherap* or CBT or intervention* or treatment* or 
deliver* or technique™ or training)).tw,kw,dv. (178265) 

95 or/79-94 (998502) 

96 74 and 78 and 95 (6982) 

97 (eCBT* or iCBT* or cCBT*).tw,kw,dv. (2326) 

98 74 and 97 (1512) 

99 (MoodGym or Big White Wall or Beating the Blues or Fear Fighter or E compass or 
Ecompass or Deprexis or Moodkit or Living Life to the Full).tw,kw,dv. (409) 

100 96 or 98 or 99 (7346) 

101 Economics/ (276061) 

102 Health Economics/ or Pharmacoeconomics/ or Drug Cost/ or Drug Formulary/ (130901) 
103 Economic Aspect/ or exp Economic Evaluation/ (424121) 

104 (econom* or price or prices or pricing or priced or discount* or expenditure* or budget* or 
pharmacoeconomic* or pharmaco-economic’).tw,kw. (962778) 

105 exp "Cost"/ (548008) 

106 (cost or costs or costing or costly).ti. (252770) 

107 cost effective*.tw,kw. (305828) 

108 (cost* adj2 (util* or efficac* or benefit* or minimi* or analy* or saving* or estimate” or 
allocation or control or sharing or instrument* or technolog*)).ab,kw. (206718) 

109 Monte Carlo Method/ (57252) 

110 (decision adj1 (tree* or analy* or model*)).tw,kw. (48842) 

111 (markov or markow or monte carlo).tw,kw. (125756) 

112 Quality-Adjusted Life Years/ (34469) 

113 (QOLY or QOLYs or HRQOL or HRQOLs or QALY or QALYs or QALE or QALEs).tw,kw. 
(68696) 

114 ((adjusted adj2 (quality or life)) or (willing* adj2 pay) or sensitivity analys*s).tw,kw. 
(123838) 

115 or/101-114 (2176900) 

116 100 and 115 (1497) 

117 Case Report/ or Comment/ or Editorial/ or Letter/ or conference abstract.pt. (9640928) 
118 116 not 117 (1417) 

119 limit 118 to english language [Limit not valid in CDSR; records were retained] (1393) 
120 limit 119 to yr="2000 -Current" (1344) 

121 120 use emez (352) 

122 "depression (emotion)"/ (130566) 

123 major depression/ (162568) 

124 recurrent depression/ (750) 

125 treatment resistant depression/ (4805) 

126 (depress* or MDD).ti,ab,id. (1218676) 

127 anxiety/ (295392) 

128 anxiety disorders/ (57570) 

129 generalized anxiety disorder/ (11394) 

130 panic disorder/ (26885) 

131. panic attack/ (29198) 

132 social anxiety/ (75770) 

133 exp phobias/ (23810) 

134 separation anxiety disorder/ (5684) 

135 Elective Mutism/ (1492) 


Ontario Health Technology Assessment Series; Vol. 19: No. 6, pp. 1-199, February 2019 152 


Appendices February 2019 


136 (anxiet* or panic* or phobi* or agoraphobi* or GAD or mute or mutism).ti,ab,id. (644312) 
137 or/122-136 (1702449) 

138 cognitive behavior therapy/ (42695) 

139 cognitive therapy/ (82916) 

140 (((cognitive or behavio*) adj2 (therap* or psychotherap”*)) or cognitive behavio* or 
cognition therap* or CBT*).ti,ab,id. (158916) 

141 or/138-140 (191538) 

142 Internet/ (189187) 

143 Computer Assisted Therapy/ (12560) 

144 Computer Mediated Communication/ (5262) 

145 computer assisted instruction/ (91414) 

146 computer software/ (106540) 

147 computer applications/ (11097) 

148 mobile devices/ (1868) 

149 electronic communication/ (3045) 

150 human computer interaction/ (15139) 

151. Information Technology/ (16974) 

152 electronic learning/ (662) 

153 online therapy/ (2372) 

154 Telemedicine/ (40471) 

155 (internet* or Beacon or app or apps or (mobile adj2 application*) or smartphone* or smart 
phone* or mobile based or e mail* or email* or electronic mail* or "Information and 
communication technology" or "Information and communication technologies" or emedicine or e 
medicine or ehealth* or e health* or emental health* or e mental health* or etherap* or e therap* 
or epsychiatr* or e psychiatr* or epsychol* or e psychol* or telemedicine or tele medicine or 
telehealth* or tele health* or telemental health* or tele mental health* or telecare or tele care or 
teletherap* or tele therap* or telepsychiatr* or tele psychiatr* or telepsychol* or tele 
psychol*).ti,ab,id. (287368) 

156 ((technolog* or computer* or digital* or webbased or web based or webdeliver* or web 
deliver* or online) adj6 (therap* or psychotherap* or CBT or intervention* or treatment* or 
deliver* or technique™ or training)).ti,ab,id. (173133) 

157 or/142-156 (767401) 

158 137 and 141 and 157 (5986) 

159 (eCBT* or iCBT* or cCBT").ti,ab,id. (2264) 

160 137 and 159 (1441) 

161 (MoodGym or Big White Wall or Beating the Blues or Fear Fighter or E compass or 
Ecompass or Deprexis or Moodkit or Living Life to the Full).ti,ab,id. (894) 

162 158 or 160 or 161 (6342) 

163 economics/ or economy/ (373151) 

164 pharmacoeconomics/ or health care economics/ (176143) 

165 (econom* or price or prices or pricing or priced or discount* or expenditure* or budget* or 
pharmacoeconomic* or pharmaco-economic’).tw. (938408) 

166 exp "costs and cost analysis"/ (572001) 

167 cost*.ti. (273267) 

168 cost effective*.tw. (301920) 

169 (cost* adj2 (util* or efficacy* or benefit* or minimi* or analy* or saving* or estimate* or 
allocation or control or sharing or instrument* or technolog’*)).ab,id. (198711) 

170 markov chains/ (18227) 

171 (decision adj1 (tree* or analy* or model*)).tw. (42927) 

172 (markov or markow or monte carlo).tw. (123251) 
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173 (QOLY or QOLYs or HRQOL or HRQOLs or QALY or QALYs or QALE or QALEs).tw 
(68076) 

174 ((adjusted adj2 (quality or life)) or (willing* adj2 pay) or sensitivity analys*s).tw. (122344) 
175 or/163-174 (2091190) 

176 162 and 175 (955) 

177 (editorial or comment reply or letter).dt. (182549) 

178 176 not 177 (950) 

179 limit 178 to english language [Limit not valid in CDSR; records were retained] (927) 
180 limit 179 to yr="2000 -Current" (912) 

181 180 use psyh (154) 

182 57 or 121 or 181 (939) 

183 182 use ppez (241) 

184 182 use coch (0) 

185 182 use cctr (173) 

186 182 use clhta (2) 

187 182 use cleed (17) 

188 182 use emez (352) 

189 182 use psyh (154) 

190 remove duplicates from 182 (525) 


CINAHL 

# Query Results 

S1__| (MH "Depression") 81,441 

S2_ | depress* OR MDD 135,892 

S3 | (MH "Anxiety") 30,504 

S4_ | (MH "Anxiety Disorders") 8,372 

S5_ | (MH "Generalized Anxiety Disorder") 290 

S6 | (MH "Panic Disorder") 1,848 

S7_ | (MH "Phobic Disorders+") 4,374 

S8 | (MH "Separation Anxiety") 547 

S9_ | (MH "Mutism") 223 
anxiet* OR anxious* OR panic* OR phobi* OR agoraphobi* OR GAD OR mute 

$10 | OR mutism 75,954 

$11) S1 OR S2 OR S3 OR S4 OR S5 OR S6 OR S7 OR S8 OR S9 OR S10 178,448 

$12 | (MH "Cognitive Therapy") 15,083 
((cognitive OR behavio*) N2 (therap* OR psychotherap*)) OR cognitive 

$13 | behavio* OR cognition therap* OR CBT* 39,254 

$14 | S12 OR S13 39,254 

$15 | (MH "Internet") 38,889 
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S16 | (MH "World Wide Web") 63,859 
$17 | (MH "World Wide Web Applications") 4,685 
$18 | (MH "Therapy, Computer Assisted") 4,739 
$19 | (MH "Mobile Applications") 2,936 
$20 | (MH "Telehealth") 5,036 
$21 | (MH "Telemedicine") 7,602 
$22 | (MH "Telepsychiatry") 232 
$23 | (MH "Remote Consultation") 1,485 


internet* OR Beacon OR app OR apps OR (mobile N2 application*) OR 
smartphone* OR smart phone* OR mobile based OR e mail* OR email* OR 
electronic mail* OR "Information and communication technology" OR 
"Information and communication technologies" OR emedicine OR e medicine 
OR ehealth* OR e health* OR emental health* OR e mental health* OR 
etherap* OR e therap* OR epsychiatr* OR e psychiatr* OR epsychol* OR e 


$24 | psychol* 72,643 
telemedicine OR tele medicine OR telehealth* OR tele health* OR telemental 
health* OR tele mental health* OR telecare OR tele care OR teletherap* OR 
tele therap* OR telepsychiatr* OR tele psychiatr* OR telepsychol* OR tele 
$25 | psychol* 14,535 
((technolog* OR computer* OR digital* OR webbased OR web based OR 
webdeliver* OR web deliver* OR online) N6 (therap* OR psychotherap* OR 
$26 | CBT OR intervention* OR treatment* OR deliver* OR technique* OR training)) 24,533 
S15 OR S16 OR S17 OR S18 OR S19 OR S20 OR S21 OR S22 OR S23 OR 
$27 | S24 OR S25 OR S26 165,807 
$28 | S11 AND S14 AND S27 997 
$29 | eCBT* or iCBT* or cCBT* 205 
$30 | S11 AND S29 148 
MoodGym OR Big White Wall OR Beating the Blues OR Fear Fighter OR E 
S31 | compass OR Ecompass OR Deprexis OR Moodkit OR Living Life to the Full 144 
$32 | S28 OR S30 OR S31 1,123 
$33 | (MH "Economics") 11,576 
$34 | (MH "Economic Aspects of Illness") 7,284 
S35 | (MH "Economic Value of Life") 533 
S36 | MH "Economics, Dental" 110 
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S37 | MH "Economics, Pharmaceutical" 1,833 
S38 | MW "ec" 148,396 


(econom* or price or prices or pricing or priced or discount* or expenditure” or 


$39 | budget* or pharmacoeconomic* or pharmaco-economic”*) 230,982 
S40 | (MH "Costs and Cost Analysis+") 90,943 
$41 | TI cost* 42,599 
S42 | (cost effective) 32,110 
AB (cost* N2 (util* or efficacy* or benefit* or minimi* or analy* or saving* or 
$43 | estimate* or allocation or control or sharing or instrument* or technolog*)) 22,836 
$44 | (decision N1 (tree* or analy* or model*)) 5,873 
S45 | (markov or markow or monte carlo) 4,053 
S46 | (MH "Quality-Adjusted Life Years") 3,210 
(QOLY or QOLYs or HRQOL or HRQOLs or QALY or QALYs or QALE or 
S47 | QALEs) 7,837 
$48 | ((adjusted N1 (quality or life)) or (willing* N2 pay) or sensitivity analys?s) 14,000 
$33 OR S34 OR S35 OR S36 OR S37 OR S38 OR S39 OR S40 OR S41 OR 
S49 | S42 OR S43 OR S44 OR S45 OR S46 OR S47 OR S48 310,969 
$50 | S32 AND S49 144 
$51 | PT Case Study or Commentary or Editorial or Letter or Proceedings 441,336 
$52 | S50 NOT S51 141 
$50 NOT S51 
$53 | Limiters - Published Date: 20000101-; English Language 139 


Search for Intervention-Related Health State Utilities 
Search date: February 28, 2018 


Database: All Ovid MEDLINE(R) <1946 to Present> 


Search Strategy 


1 Cognitive Therapy/ (21328) 
2 (((cognitive or behavio*) adj2 (therap* or psychotherap*)) or cognitive behavio* or cognition 
therap* or CBT*).ti,ab,kf. (86053) 

3 1 or 2 (43955) 

4 Internet/ (63558) 

5 Therapy, Computer-Assisted/ (6202) 
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6 Computer-Assisted Instruction/ (11053) 
7 Mobile Applications/ (2672) 
8 Telemedicine/ (16845) 
9 Remote Consultation/ (4334) 
10 (internet* or Beacon or app or apps or (mobile adj2 application*) or smartphone* or smart 
phone* or mobile based or e mail* or email* or electronic mail* or "Information and 
communication technology" or "Information and communication technologies" or emedicine or e 
medicine or ehealth* or e health* or emental health* or e mental health* or etherap* or e therap* 
or epsychiatr* or e psychiatr* or epsychol* or e psychol* or telemedicine or tele medicine or 
telehealth* or tele health* or telemental health* or tele mental health* or telecare or tele care or 
teletherap* or tele therap* or telepsychiatr* or tele psychiatr* or telepsychol* or tele 
psychol*).ti,ab,kf. (98140) 
11 ((technolog* or computer or digital* or webbased or web based or webdeliver* or web 
deliver* or online) adj6 (therap* or psychotherap* or CBT or intervention* or treatment* or 
deliver* or technique” or training)).ti,ab,kf. (62759) 
12 or/4-11 (215025) 
13 3 and 12 (3242) 
14 (eCBT* or cCBT* or iCBT*).ti,ab,kf. (634) 
15 (MoodGym or Big White Wall or Beating the Blues or Fear Fighter or E compass or 
Ecompass or Deprexis or Moodkit or Living Life to the Full).ti,ab,kf. (111) 
16 13 o0r 14 or 15 (8521) 
17 Quality-Adjusted Life Years/ (9861) 
18 (quality adjusted or adjusted life year*).tw. (12827) 

(qaly* or qald* or qale* or qtime*).tw. (8263) 

(illness state$1 or health state$1).tw. (5340) 
21 (hui or huil or hui2 or hui3).tw. (1231) 

(multiattrioute* or multi attribute*).tw. (729) 
23 (utility adj3 (score$1 or valu* or health* or cost* or measure* or disease* or mean or gain or 
gains or index*)).tw. (11528) 
24 utilities.tw. (5792) 
25 (eq-5d or eq5d or eq-5 or eq5 or euro qual or euroqual or euro qual5d or euroqual5d or 
euro gol or eurogol or euro gol5d or eurogol5d or euro quol or euroquol or euro quol5d or 
euroquol5d or eur gol or eurgol or eur gol5d or eurgol5d or euro?qul or eur?qul5d or euro* 
quality of life or European qol).tw. (8210) 
26 (euro* adj3 (5 d or 5d or 5 dimension* or 5dimension* or 5 domain* or 5domain*)).tw. 
(2817) 
27 (sf36* or sf 36* or sf thirtysix or sf thirty six).tw. (18845) 
28 (time trade off$1 or time tradeoff$1 or tto or timetradeoff$1).tw. (1610) 
29 ((qol or hrgol or quality of life).ti. or *quality of life’) and ((qol or hrgqol* or quality of life) adj2 
(increas* or decreas* or improve* or declin* or reduc* or high* or low* or effect or effects of 
worse or score or scores or change$1 or impact$1 or impacted or deteriorate$)).ab. (25226) 
30 Cost-Benefit Analysis/ and (cost effectiveness ratio* and (perspective™ or life 
expectanc*)).tw. (2682) 
*quality of life/ and (quality of life or qol).ti. (45114) 
32 quality of life/ and ((quality of life or gol) adj3 (improve* or chang’*)). 
33 quality of life/ and ((quality of life or gol) adj (score$1 or measure$1 
34 quality of life/ and health-related quality of life.tw. (24914) 
35 quality of life/ and ec.fs. (8752) 
36 quality of life/ and (health adj3 status).tw. (7471) 
37 (quality of life or gol).tw. and cost-benefit analysis/ (10102) 
38 models, economic/ (8557) 


oo 
— 
— 


— 


tw. (19685) 
)).tw. (9722) 
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39 or/17-38 (130798) 
40 16 and 39 (158) 
41 limit 40 to english language (157) 


Grey Literature Search 
Search dates: February 9-14, 2018 


Websites searched: HTA Database Canadian Repository, Alberta Health Technologies 
Decision Process reviews, Canadian Agency for Drugs and Technologies in Health (CADTH), 
Institut national d’excellence en santé et en services sociaux (INESSS), Institute of Health 
Economics (IHE), McGill University Health Centre Health Technology Assessment Unit, 
National Institute for Health and Care Excellence (NICE), Agency for Healthcare Research and 
Quality (AHRQ) Evidence-based Practice Centers, Australian Government Medical Services 
Advisory Committee, Centers for Medicare & Medicaid Services Technology Assessments, 
Institute for Clinical and Economic Review, Ireland Health Information and Quality Authority 
Health Technology Assessments, Washington State Health Care Authority Health Technology 
Reviews, PROSPERO International prospective register of systematic reviews, Tufts Cost- 
Effectiveness Analysis Registry 


Keywords used: Internet, computer’, online, digital*, webbased, web based, app, apps, 
smartphone”, smart phone*, CBT, behavioral therapy, behavioural therapy, behavior therapy, 
behaviour therapy, cognitive therapy, psychotherapy, psychotherapies 


Results: 13 (17 PROSPERO systematic review protocols not counted in PRISMA) 
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Appendix 2: Critical Appraisal of Clinical Evidence 
Table A1: Risk of Bias? Among Systematic Reviews (ROBIS Tool) 


February 2019 


Phase 2 Phase 3 
Identification 
Study Eligibility and Selection of Data Collection and Synthesis and 

Author, Year Criteria Studies Study Appraisal Findings Risk of Bias in the Review 
Arnberg et al, 20147° High? Low Low Low Low 
Adelman et al, 2014” Low Low Low Low Low 
Andrews et al, 20187! Low Low Low Low Low 
Richards et al, 201574 Low Low Low Low Low 
Kampmann et al, Low Low High® Low High 
20167 

Dedert et al, 20137° High? Low Low Low Low 
Kaltenthaler et al, Low High? High® Low Low 
200872 


Abbreviation: ROBIS, Risk of Bias in Systematic Reviews. 
Possible risk of bias levels: low, high, unclear. 
Potential bias due to language restrictions. 


°Potential bias as additional methods to database searching were not used. Unclear how passive and active control is defined. 


4Potential bias due to single reviewer for data extraction. 
°Unicear if double reviewer for data extraction. 
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Appendix 3: Selected Excluded Studies 


Citation Primary Reason for Exclusion 

Andersson et al, 20148 No subgroup analysis of clinical diagnosis studies 

Carlbring et al, 201819 No subgroup analysis of clinical diagnosis studies 

Olthuis et al, 2016'4° No details on mild to moderate major depression 

Baumeister et al, 201414 Unclear if iCBT programs were included in subgroup analysis 
Karyotaki et al, 2017% No details on mild to moderate major depression 
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Appendix 4: Results of Applicability and Limitation Checklists for Studies 
Included in Economic Literature Review 


Table A2: Assessment of the Applicability of Studies Assessing the Cost-Effectiveness of iCBT for 
Major Depression 


Is the health care 
system in which 


Are estimates 


the study was of relative 
Is the study Are the conducted Were the treatment 
population interventions sufficiently similar perspectives effect from the 
Author, Year, similar to the similar to the to the Ontario clearly stated? best available 
Country question? question? context? What were they? source? 
Duarte et al, 2017,°” Yes Yes, iCBT plus No Yes, NHS Unclear 
Littlewood, et al, usual care (drug 
2015,°° United allowed) vs usual 
Kingdom care, low-guided 
Romero-Sanchiz et Yes Yes, No Yes, Spain, Unclear 
al, 2017,*° Spain unguided/guided societal 
vs TAU 
Lee et al, 2017,*° Yes Partially No Yes, health care Unclear 
Australia payer 
Brabyn et al, 2016,°° Yes Partially No Yes, NHS andPPS Unclear 
United Kingdom 
Dixon et al, 2016,° Yes Partially No Yes, NHS andPPS Unclear 
United Kingdom 
Solomon et al, Yes Partially No No, health care Unclear 
2015,**Australia payer 
Titov et al, 2015, Yes Partially No No, health care Unclear 
Australia payer 
Gerhards et al, Yes Partially No Yes, employer and _—_ Unclear 
2015,*° Netherlands societal 
Phillips et al, 2014," Partially Partially No Yes, societal Unclear 
United Kingdom 
Gerhards et al, Yes Partially No Yes, societal Unclear 
2010,*° Netherlands 
Warmerdam et al, Yes Partially No Yes, societal Unclear 
2010,*” Netherlands 
Hollinghurst et al, Yes Partially No Yes, societal Unclear 
2010,*° United 
Kingdom 
Kaltenthaler et al, Yes Partially No Yes, NHS Unclear 
2006,°° United 
Kingdom 
McCrone et al, Partially Partially No Yes, NHS Unclear 
2004,” United 
Kingdom 
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Are all future costs 
and outcomes 


Is the value of 
health effects 


Are costs and 
outcomes from other 
sectors fully and 


February 2019 


Overall judgement (directly 


discounted? expressed in terms appropriately applicable/partially 

Author, Year, (If yes, at what of quality-adjusted measured and applicable/ 
Country rate?) life-years? valued? not applicable) 
Duarte et al, Yes (3.5%) Yes Yes, microcosting Partially applicable 
2017,°” approach accounting 
Littlewood, et al, for cost of CBT, 
2015,°6 United medications, health 
Kingdom and social care 

services 
Romero-Sanchiz No, 12 mo, NA Yes Yes, microcosting Partially applicable 
et al, 2017, approach accounting 
Spain for both direct and 

indirect costs 
Lee et al, 2017,“° No, 12 mo, NA Yes Yes Partially applicable 
Australia 
Brabyn et al, No, 12 mo, NA Yes Yes Partially applicable 
2016,°° United 
Kingdom 
Dixon et al, No, 12 mo, NA Yes Yes Partially applicable 
2016,°° United 
Kingdom 
Solomon et al, No, 6 mo, NA Yes Partially Partially applicable 
2015,** Australia 
Titov et al, No, 8 wk, NA Yes Partially Partially applicable 
2015," Australia 
Gerhards et al, No, 12 mo, NA Yes Partially Partially applicable 
2015, 
Netherlands 
Phillips et al, No, 12 mo, NA Yes Yes Partially applicable 
2014," United 
Kingdom 
Gerhards et al, No, 12 mo, NA Yes Yes Partially applicable 
2010,%° 
Netherlands 
Warmerdam et No, 12 wk, NA Yes Unclear Partially applicable 
al, 2010,*” 
Netherlands 
Hollinghurst et al, No, 8mo, NA Yes Unclear Partially applicable 
2010,*° United 
Kingdom 
Kaltenthaler et al, No, 18 mo, NA Yes Yes Partially applicable 
2006,°° 
United Kingdom 
McCrone et al, No, 8 mo, NA No Yes Partially applicable 


2004,*? United 
Kingdom 


Abbreviations: CBT, cognitive-behavioural therapy; iCBT, internet-delivered CBT; NA, not applicable; NHS, National Health Service; PPS, personal 
social services; TAU, treatment as usual. 
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Table A3: Assessment of the Limitations of Studies Assessing the Cost-Effectiveness of iCBT for 


Major Depression 


Do the 
Does the estimates 
model Is the time Are the of relative Are all 
structure horizon estimates treatment important Are the 
adequately sufficiently Are all of relative effect and estimates of 
reflect the long to important treatment match the relevant resource 
nature of the reflect all and effects estimates (direct) use 
health important relevant obtained contained costs obtained 
condition differences health from best in the included from best 
Author, Year, under incostsand outcomes available clinical in the available 
Country evaluation? outcomes? included? sources? report? analysis? sources? 
Duarte et al, NA Partially Yes Unclear NA Yes Unclear 
2017,°” 
Littlewood, et al, 
2015,°° United 
Kingdom 
Romero-Sanchiz NA No Yes Unclear NA Yes Unclear 
et al, 2017, 
Spain 
Lee et al, 2017,4° No No Partly Unclear No Yes Unclear 
Australia 
Brabyn et al, NA No Partly Unclear No Yes Unclear 
2016,° United 
Kingdom 
Dixon et al, NA No Partly Unclear No Yes Unclear 
2016,°° United 
Kingdom 
Solomon et al, Partially No No Unclear NA Yes Unclear 
2015, Australia 
Titov et al, NA No No Unclear NA Yes Unclear 
2015," Australia 
Gerhards et al, NA No Yes Unclear Yes Unclear Unclear 
2015, 
Netherlands 
Phillips et al, NA No No Unclear NA Yes Unclear 
2014,*' United 
Kingdom 
Gerhards et al, NA No No Unclear NA Yes Unclear 
2010,” 
Netherlands 
Warmerdam et NA No No Unclear NA Yes Unclear 
al, 2010,*” 
Netherlands 
Hollinghurst et al, NA No No Unclear NA Yes Unclear 
2010,*° United 
Kingdom 
Kaltenthaler et al, Partially No No Unclear NA Yes Unclear 
2006,°° United 
Kingdom 
McCrone et al, NA No No Unclear No Yes Unclear 
2004,*? United 
Kingdom 
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Is an 
appropriate Are all 
incremental important and _ Overall assessment 
Are the unit analysis uncertain including applicability 
costs of presented, or parameters to the project 
resources can it be subjected to (minor limitations/ 
obtained from _ calculated from appropriate Is there a potentially serious 
Author, Year, best available the reported sensitivity potential conflict _limitations/very serious 
Country sources? data? analysis? of interest? limitations) 
Duarte et al, Yes Yes, multiple Yes Unclear Minor limitations, 
2017,°” imputation and interventions may be 
Littlewood, et al, adjustment from contaminated, 
2015,°° United baseline decreasing the effect of 
Kingdom cCBT, pragmatic trial 
Romero-Sanchiz Yes Yes, ITT Yes Unclear Minor limitations, 
et al, 2017, pragmatic trial 
Spain 
Lee et al, 2017,*9  ~— Unclear No Unclear Unclear Potentially serious 
Australia limitations 
Brabyn et al, Yes Yes, multiple Yes Unclear Minor limitations, RCT, 
2016,°8 United imputation not long-term 
Kingdom 
Dixon et al, Yes Yes Yes Unclear Potentially serious 
2016,°° United limitations due to large 
Kingdom amounts of missing data, 
analyses are 
comprehensive and 
accounting for it, but 
results are uncertain 
Solomon et al, Yes Yes Yes Unclear Potentially serious 
2015,** Australia limitations, no clear 
model structure, paper 
not well written, short 
time horizon, number of 
limitations 
Titov et al, Yes No Yes Unclear Potentially serious 
2015, Australia limitations, small study 
Gerhards et al, Yes No Unclear Unclear Minor limitation to HE 
2015, analysis, but selected 
Netherlands population 
Phillips et al, Yes No Yes Unclear Major limitations 
2014,*' United 
Kingdom 
Gerhards et al, Unclear Partially Yes Unclear Potentially serious 
2010,%° limitations, small trial, not 
Netherlands sure how they allocated, 
short-term, no INB 
Warmerdam et Yes No Yes Unclear Potentially serious 
al, 2010,*” limitations 
Netherlands 
Hollinghurst et al, Yes No Yes Unclear Minor limitations, RCT, 
2010,*° United not long-term 
Kingdom 
Kaltenthaler et al, Yes Partially Yes Unclear Potentially serious 
2006,°° United limitations 
Kingdom 
McCrone et al, Yes No Yes Unclear Potentially serious 


2004,*? United 
Kingdom 


limitations 


Abbreviations: CBT, cognitive-behavioural therapy; iCBT, internet-delivered CBT; HE, health economics; INB, incremental net benefit; 
ITT, intention-to-treat analysis; NA, not applicable; RCT, randomized controlled trial. 
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Table A4: Assessment of the Applicability of Studies Assessing the Cost-Effectiveness of iCBT for 
Anxiety Disorders 


Is the health care 


system in which the 


Are estimates 


study was of relative 
Is the study Are the conducted Were the treatment 
population interventions sufficiently similar perspectives effect from the 
Author, Year, similar to the similar to the to the current clearly stated? best available 
Country question? question? Ontario context? What were they? source? 
Kumar et al, Yes, GAD Yes No Yes, health care Partially 
2018, United payer and societal 
States 
EI Alaoui et al, Yes, SAD Yes No Yes, health care NA 
2017,°' Sweden payer 
Hedman et al, Yes, SHA Yes No Yes, societal Partially 
2016,°° Sweden 
Dear et al, 2015, = Partially, older Yes No Yes, health sector Unclear 
Australia age anxiety 
Nordgren et al, Partially, anxiety Yes No Yes, societal Unclear 
2014,°° Sweden including 
comorbid 
Hedman et al, Yes, SAD Yes No Yes, societal Unclear 
2014,°° Sweden 
Hedman et al, Yes, SHA Yes No Yes, societal Unclear 
2013,5” Sweden 
National Yes, SAD Yes No Yes, NHS andPPS Partially 
Collaborating 
Centre for Mental 
Health and NICE, 
2013, United 
Kingdom 
Joesch et al, Partially, Partially, iCBT No Yes, health care Unclear 
2012,5> United collaborative embedded in payer 
States care, sick at collaborative care 
baseline program 
Hedman et al, Yes, SAD Yes No Yes, societal Unclear 
2011,°° Sweden 
National Yes, panic Yes No Yes, NHS & PPS Unclear 
Collaborating 
Centre for Mental 
Health and NICE, 
2011,° United 
Kingdom 
Bergstrom et al, Yes, panic Yes No Yes, societal Unclear 
2010,°* Sweden 
Titov et al, 2009,°° Yes, SAD Partially No Yes, health sector No 
Australia 
McCrone et al, Yes, panic Partially No Unclear No 
2009,°? Sweden 
Mihalopoulos et al, Yes, panic Yes No Yes, health sector Unclear 
2005,*' Australia 
Kaltenthaler et al, Yes, panic Yes No Yes, NHS Unclear 
2006,°° United 
Kingdom 
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Are costs and 
Are all future costs Is the value of outcomes from other 
and outcomes health effects sectors fully and Overall judgement (directly 
discounted? expressed in terms appropriately applicable/partially 
Author, Year, (If yes, at what of quality-adjusted measured and applicable/ 
Country rate?) life-years? valued? not applicable) 


Kumar et al, Yes, 3% Yes Yes Partially applicable 
2018, United 
States 


EI Alaoui et al, Yes, 0% (38% to5% in NA Yes Partially applicable 
2017," sensitivity analysis) 
Sweden 


Hedman et al, No, 12 wks Yes (data not Yes Partially applicable 
2016, presented) 
Sweden 


Dear et al, No, 2 mo Yes Yes Partially applicable 
2015,” 
Australia 


Nordgren et al, | No, 10 wks Yes Yes Partially applicable 
2014,°° 
Sweden 


Hedman et al, Unclear Yes Yes Partially applicable 
2014,°° 
Sweden 


Hedman et al, No Yes Yes Partially applicable 
2013,°” 
Sweden 


National Unclear Yes Yes Partially applicable 
Collaborating 

Centre for 

Mental Health 

and NICE, 

2013, 

United 

Kingdom 


Joesch et al, No Yes Yes Partially applicable 
2012,°° United 
States 


Hedman et al, Unclear Yes Yes Partially applicable 
2011,5 
Sweden 


National No, 1 yr No Yes Partially applicable 
Collaborating 

Centre for 

Mental Health 

and NICE, 

2011,° 

United 

Kingdom 

Bergstrom et No No Yes Partially applicable 


al, 2010,°4 
Sweden 


Titov et al, No No Partially Partially applicable 
2009,°° 
Australia 


McCrone et al, No No Unclear Partially applicable 
2009,°7 
Sweden 


Mihalopoulos No No Unclear Partially applicable 
et al, 2005,°! 
Australia 
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Are costs and 


Are all future costs Is the value of outcomes from other 
and outcomes health effects sectors fully and Overall judgement (directly 
discounted? expressed in terms appropriately applicable/partially 
Author, Year, (If yes, at what of quality-adjusted measured and applicable/ 
Country rate?) life-years? valued? not applicable) 
Kaltenthaler et No, 1 yr Yes Unclear Partially applicable 
al, 2006,° 
United 
Kingdom 


Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; GAD, generalized anxiety disorder; NA, not applicable; NHS, 
National Health Service; NICE, National Institute for Health and Care Excellence; PPS, personal social services; SAD, social anxiety disorder; SHA, 
social health anxiety. 
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Table A5: Assessment of the Limitations of Studies Assessing the Cost-Effectiveness of iCBT for 


Anxiety Disorders 


Does the Do the 
model Are the estimates of Are all Are the 
structure estimates relative important estimates 
adequately Are all of relative treatment and of 
reflect the important treatment — effect match relevant resource 
nature of the Is the time horizon and effects the (direct) use 
health sufficiently long to relevant obtained estimates costs obtained 
condition reflect all important health from best contained in included from best 
under differences in costs outcomes available the clinical in the available 
Author, Year, Country evaluation? and outcomes? included? sources? report? analysis? sources? 
Kumar et al, 2018, Partially Yes, lifetime Partially, Partially No Yes Unclear 
United States (assumption made no 
on 5-yr efficacy of recurrence 
iCBT) 
EI Alaoui et al, 2017,°" NA No NA NA No Yes Unclear 
Sweden 
Hedman et al, 2016, NA No Unclear Partially No Yes Unclear 
Sweden 
Dear et al, 2015,°2 NA No Partially, Unclear No Yes Yes 
Australia no relapse 
Nordgren et al, 2014,°° NA No No Unclear No Yes Unclear 
Sweden 
Hedman et al, 2014,°9 NA Partially Yes Unclear No Yes Unclear 
Sweden 
Hedman et al, 2013,°7 NA No No Unclear No Unclear Unclear 
Sweden 
National Collaborating Partially Yes Yes Partially Unclear Yes Yes 
Centre for Mental (assumption made 
Health and NICE, on long-term 
2013,” United efficacy of iCBT) 
Kingdom 
Joesch et al, 2012,°° NA No Yes Unclear No Yes Unclear 
United States 
Hedman et al, 2011,°8 NA Partially Yes Unclear No Yes Unclear 
Sweden 
National Collaborating Partially No No Unclear No Yes Partially 
Centre for Mental 
Health and NICE, 
2011,° United Kingdom 
Bergstrom et al, NA No No Unclear No Partially Unclear 
2010,°* Sweden 
Titov et al, 2009,° NA No No No No No No 
Australia 
McCrone et al, 2009, NA No No No No No No 
Sweden 
Mihalopoulos et al, NA No No Unclear No Unclear Unclear 
2005,°' Australia 
Kaltenthaler et al, Partially No No Unclear No Partially Unclear 
2006,*° United 
Kingdom 
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Author, Year 


Are the unit 
costs of 
resources 
obtained from 
best available 
sources? 


Is an 
appropriate 
incremental 

analysis 
presented or 
can it be 
calculated 
from the 
reported data? 


Are all 
important and 
uncertain 
parameters 
subjected to 
appropriate 
sensitivity 
analysis? 


Is there a 
potential 
conflict of 
interest? 


February 2019 


Overall assessment 
including 
applicability to the 
project 
(Minor limitations/ 
potentially serious 
limitations/very 
serious limitations) 


Kumar et al, 2018,°° 
United States 


EI Alaoui et al, 2017,°" 
Sweden 


Hedman et al, 2016, 
Sweden 


Dear et al, 2015, 
Australia 


Nordgren et al, 2014,°6 
Sweden 


Hedman et al, 2014,°° 
Sweden 


Hedman et al, 2013,5” 
Sweden 


National Collaborating 
Centre for Mental Health 
and NICE, 2013, United 
Kingdom 


Joesch et al, 2012,°5 
United States 


Unclear 


Unclear 


Unclear 


(self-reported 
questionnaire) 


Yes 


Unclear 


Unclear 


Yes 


Partially 


Unclear 


No 


Yes 


Yes 


Yes 


Yes 


Yes 


Unclear 


Partially 


Yes 


No 


Unclear 


No 


No 


Unclear 


Unclear 


Partially 


Partially 


Unclear 


Unclear 


Unclear 


Unclear 


Unclear 


Unclear 


Unclear 


Unclear 


Unclear 


Unclear 


Potentially serious 
limitations 


Potentially serious 
limitations due to CMA 
design and duplication 
of data analysis 


Potentially serious 
limitations, selected 
sample, self-reported 
costs, no sensitivity 
analysis, no CEAC, no 
all comparators, no 
clear influence of 
medications 


Potentially serious 
limitations 


Potentially serious 
limitations, selected 
population, only TAU, 
Unclear estimation of 
QALYs, very large 
increment as 
compared to other 
studies 


Potentially serious 
limitations, selected 
population, not clear 
cost calculations, only 
group CBT, drug 
comparator probably 
not used for this 
condition 


Potentially serious 
limitations, short 
duration, estimates not 
presented well 


Minor limitations, large 
modeling assumptions, 
but all strategies are 
modeled, results for 
year 1 are not 
presented, but are 
discussed as cost- 
effective, uncertainty 
analysis not presented 


Potentially serious 
limitations, study 
population quite sick, 
other comparators not 
well defined, iCBT 
mixed with other 
interventions 
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Author, Year 


Are the unit 
costs of 
resources 
obtained from 
best available 
sources? 


Is an 
appropriate 
incremental 

analysis 
presented or 
can it be 
calculated 
from the 
reported data? 


Are all 
important and 
uncertain 
parameters 
subjected to 
appropriate 
sensitivity 
analysis? 


Is there a 
potential 
conflict of 
interest? 


February 2019 


Overall assessment 
including 
applicability to the 
project 
(Minor limitations/ 
potentially serious 
limitations/very 
serious limitations) 


Hedman et al, 2011,°8 
Sweden 


National Collaborating 
Centre for Mental Health 
and NICE, 2011,° United 
Kingdom 


Bergstrom et al, 2010, 
Sweden 


Titov et al, 2009,° 
Australia 


McCrone et al, 2009,°2 
Sweden 


Mihalopoulos et al, 
2005,*" Australia 


Unclear 


Partly 


Unclear 


No 


No 


Unclear 


Yes 


Yes 


No 


No 


No 


No 


Unclear 


Unclear 


No 


No 


Unclear 


No 


Unclear 


Unclear 


Unclear 


Unclear 


Unclear 


Unclear 


Potentially serious 
limitations, selected 
population, no clear 
cost calculations, only 
group CBT, drug 
comparator probably 
not used for this 
condition, not clear 
quality of this RCT 


Potentially serious 
limitations, simple 
decision tree analysis, 
short modeling time, 
costs well estimated 
but may not be 
applicable to Ontario, 
utilities—EQ-5D but 
change based on 
assumption and linear 
extrapolation due to 
lack of data 


Potentially serious 
limitations, study 
population selected, 
no QALY, costs not 
well estimated, 
analysis not well done 


Very serious 
limitations, selected 
population, cost 
estimates not well 
done 


Very serious 
limitations, costs not 
measured, QALYs not 
estimated, selected 
population 


Very serious 
limitations, unclear 
how costs or outcomes 
were measured, 
QALYs not estimated, 
pilot project 


Kaltenthaler et al, 2006,°° 
United Kingdom 


Unclear 


Partially 


Unclear 


Potentially serious 
limitations, simple 
model, 1-yr time 
horizon, utilities—EQ- 
5D but changed based 
on assumption and 
linear extrapolation 
due to lack of data 
from a study in 
individuals with prior 
panic disorder 


Abbreviations: CBT, cognitive-behavioural therapy; CEAC, cost-effectiveness acceptability curve; iCBT, internet-delivered CBT; NA, not applicable; 
NICE, National Institute for Health and Care Excellence; QALY, quality-adjusted life year; RCT, randomized controlled trial; TAU, treatment as usual. 
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Appendix 5: Long-Term Cost-Effectiveness of Guided iCBT Provided Within a 
Stepped-Care Model 


Model Structure 


We combined our short-term reference case iCBT model with a previously developed 
probabilistic Markov microsimulation model’’ to examine the lifetime cost-effectiveness of 
guided iCBT by a therapist and face-to-face CBT versus usual care in people with new and 
recurrent major depression and anxiety. Our model simulates a hypothetical cohort of women 
and men aged 18 years and older with a primary diagnosis of mild to moderate major 
depression who are eligible for a course of iCBT. We use a stepped-care approach starting with 
therapist-guided iCBT followed by with group or individual face-to-face CBT if the disease recurs 
or progresses. In this analysis, we compared three strategies: (1) guided iCBT followed by 
individual CBT provided by a regulated health professional (nonphysician); (2) guided iCBT 
followed by group CBT provided by a regulated health professional (nonphysician); and 

(3) usual care that does not include CBT. We accounted for the possibility of repeating CBT up 
to three times in people with recurrent disease, as suggested by the guidelines.’*’° The model 
used a short weekly cycle to monitor changes in the progression of disease and adequately 
reflect what is being done in current clinical practice; we applied a half-cycle correction to 
balance the distribution of people who transition between health states at the beginning or end 
of each cycle. 


The model schematic is provided in Figure A1. The current model was adapted from our 
published model” and it incorporated a delivery of guided iCBT in the initial phase of a major 
depressive episode (MDE), and the benefits and costs of iCBT during all phases of the MDE. 
Prior models” included in-person CBT only, starting from the first MDE. In the current model, 
we also allow for a possibility to drop out from the iCBT or CBT course each time the courses 
were provided (i.e., initial or recurrent episodes). The input parameters for face-to-face CBT 
were derived from our prior HTA.’” The input parameters relevant to guided iCBT were based 
on the current review. All parameter values used in the current model are described in Appendix 
6 (Tables A6 to AQ). 


Our Markov model was developed through expert consultation and in accordance with the 
current clinical guidelines.”>* It captures the episodic nature of major depression and anxiety 
disorders'**:'4° and distinguishes relapse from recurrence, which are components of the natural 
course of major depression. It follows the course of major depression, which consists of the 
following: 


e Two initial phases (acute and continuation), each lasting approximately 3 months, during 
which response and relapse are monitored with the goal of achieving remission 

e The maintenance phase, lasting approximately 6-24 months, during which full recovery 
can be established or recurrence of a full MDE can occur.2>® 


Recurrence is defined as a full episode of depression after full recovery. Recurrence can 
happen in the maintenance phase 6 months after the initial diagnosis or later after a person has 
achieved a period of remission including at least 2 months with no significant symptoms.?>® 
This model accounts for factors affecting the risk of recurrence (e.g., age at disease onset and 
number of prior episodes), changes in the severity of initial disease (mild to severe major 
depression), and consequent changes in treatment options after a transition from mild to 
moderate or severe disease. It also allows for the addition of pharmacotherapy for people who 
progress to more severe health states, for medication dose increase, or medication switch for 
people whose symptoms are not improving.”” The frequency of disease progression monitoring 
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(by a general practitioner or psychiatrist) depends on the initial disease severity. In general, 
monitoring occurs every 2 weeks in the first 4 months and monthly thereafter for the remainder 
of the first year after a diagnosis of major depression. If a person achieves full remission and 
remains stable during the maintenance phase, they enter the well health state, in which they are 
considered to remain stable (without depression and receiving no treatment twice yearly follow- 
ups with a general practitioner). A patient has a chance of experiencing a recurrent episode in 
the maintenance phase, from the well health state, or while in one of the two recurrent health 
states (mild or moderate to severe, Figure 4). Our model accounted for age-dependent 
background mortality in Ontario and for differences in suicide rate based on the severity of 
major depression. '44:145 


Initial (acute) 
‘ phase: mild to 
- moderate MDE 
Maintenance phase Continuabon phase: \ 
— 


mild MDE mild MDE J moder ate/severe 
- v MDE MDE 


*Cor innuation phase: Maintenance phase! 


“\ moderate/severe 


Recurrent mild 
MODE 


: 
= Recurrent — 
moderate/severe (* 
MDE 
. 


Complex ' 
depression } @ 


Death 


Figure A1: Simplified Markov Model Schematic: Guided iCBT with a Stepped-Care Approach 


Abbreviation: MDE, major depressive episode. 

Note: Figure A1 depicts an individual-level (microsimulation) model that includes 11 Markov health states, each represented by an oval, reflecting the 
course of major depression. The simulation starts with a hypothetical person aged between 18 and 75 years with a mild or moderate major depressive 
episode. Each week cycle, the person has a chance to move among health states. Death is an absorbing Markov health state. In the acute initial phase 
lasting 12 weeks, a course of iCBT is applied. In two recurrent states, in-person CBT is the therapy of choice. People are allowed to drop out. The 
model accounts for the age at onset of major depression, counts the number of recurrent events, and modifies the risk of future major depressive 
episodes and changes in treatment accordingly. The model also takes into account the probability of death as a result of suicide according to the 
severity of disease and the probability of hospitalization in the complex depression health state. Transfer to the complex depression health state 
depends on the number of prior major depressive episodes-. 
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Appendix 6: Model Inputs Used for the Life-Time Cost-Effectiveness Analysis of 
Guided iCBT Followed by Face-to-Face CBT Versus Usual Care 


Table A6: Input Parameters Associated With the Natural History of Major Depression: 


Probabilities and Risks 


Model Parameter 


Probabilities/Rates Mean Distribution Source 
Probability of major depression, by NA Ferrari et al, 2013146 
disease severity, no prior episode 

Mild 0.68 

Moderate to severe 0.32 
Probability of major depression, by NA Hardeveld et al, 201314” 
disease severity, with prior episode 

Mild 0.24 

Moderate to severe 0.76 
Probability of dropout: NA van Ballegooijen et al, 

2014°*; Koeser et al, 
2015%8 

Guided iCBT 0.19 

Face-to-face CBT 0.16 

CBT plus pharmacotherapy 0.23 

Pharmacotherapy only 0.30 

No treatment 0.29 
Probability of being not well after 0.33 Expert consultation 
dropout 
Probability of no response, acute NA Koeser et al, 2015% 
phase 

CBT only 0.23 

CBT plus pharmacotherapy 0.18 

Pharmacotherapy only 0.70 
Probability of response (no relapse), NA Koeser et al, 2015% 
continuation phase 

CBT only 0.69 

CBT plus pharmacotherapy 0.75 

Pharmacotherapy only 0.70 

No treatment 0.43 
Annual rate of recurrence, year 1, 0.15 NA Eaton et al, 200812” 
maintenance phase 
Annual rate or recurrence, long-term NA Eaton et al, 200812” 

Year 2 0.20 

Year 4 0.75 

Year 5 0.70 

Year 10 0.50 
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Model Parameter 
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Probabilities/Rates Mean Distribution Source 
Annual rate or recurrence, long NA Hardeveld et al, 201314” 
term, after 6 months of remission# 

Year 1 0.025 

Year 2 0.045 

Year 5 0.13 

Year 10 0.23 

Year 15 0.23 

Year 20 0.42 
Rate of hospitalization 0.10 NA Health Quality Ontario, 

2016! 
Background rate of death Ontario life NA Statistics Canada, 2011'8 
tables 
Rate of death by suicide NA Khan et al, 2003145; Khan 
and Schwartz, 2007'4 

General population 0.000166 

Patients with depression 0.001 

Patients with chronic/complex 0.0024 

depression 
Risk Ratio 

Risks (95% Cl) Distribution Source 
Relative risk of recurrent major 0.96 Normal® Eaton et al, 200812’ 
depression, by age of disease onset, (0.93-0.97) (log-odds ratio) 
for each additional year 
Relative risk of recurrent MDE, for 1.18 Normal? Mueller et al, 1999149 
each additional episode (1.06-1.31) (log-odds ratio) 


Abbreviations: CBT, cognitive behavioural disorder; Cl, confidence interval; iCBT, internet-delivered CBT; MDE, major depressive episode; 


NA, not applicable. 
*Used in sensitivity scenario analysis only. 


>Distributions assigned in probabilistic sensitivity analysis. 
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Table A7: Probability of Response, Initial Phase, Mild to Moderate Major Depression: Internet- 
Delivered CBT, CBT, and Medication 


Model Parameter Mean (95% CI/SE)? SMD (95%Cl) Source 
Probability of response (improvement) 

Guided iCBT 0.73 (0.66-0.77) 0.83 (0.59-1.07) Arnberg et al, 20142° 
Individual face-to-face CBT 0.73 (0.66—0.77) NAS Dedert et al, 2013,2° 


Andrews et al, 2018,2! 
Adelman et al, 20147 


Group face-to-face CBT 0.73 (0.66-0.77) NAS Dedert et al, 2013,?° 
Andrews et al, 2018,?! 
Adelman et al, 20147 


Medication 0.70 Koeser et al, 2015% 

Abbreviations: CBT, cognitive behavioural disorder; iCBT, internet-delivered CBT; Cl, confidence interval; SMD, standardized mean difference; 

NA, not applicable; SE, standard error; SMD, standardized mean difference. 

4Beta distributions assigned in probabilistic sensitivity analysis to all parameters for which 95% Cl or SE were specified in Table A6. Two parameters 
of the beta distribution (a,8) are derived from the mean and SE (stated for each model parameter). Formulas for these calculations, derived from the 
mean and SE (provided in the Tree Age software) are: a = ([Mean‘2] x [1 — Mean])/([SE*2] - Mean); B = (1 — Mean) x ([(1 - Mean) x Mean]/([SE“2] 
- 1). 

Original value, reported in the article and clinical review. 

Reported risk of recurrence, in-person CBT vs usual care in Biesheuvel-Leliefeld et al’”® (relative risk [RR] 0.68, 95% Cl 0.53—-0.76), group CBT in 

Churchill et al9” (0.78, 95% Cl 0.69—0.93), and sertraline in Cipriani & Geddes®* (RR 0.80, 95% Cl 0.69—0.93). Details provided in our prior report.’” 


Table A8: Efficacy Estimates Used in the Economic Model: Risk of Recurrence 


Relative Risk 


Model Parameter (95% Cl) Distribution Source 

Risk of recurrence, major depression: 0.68 Normal Biesheuvel-Leliefeld 
CBT only vs no treatment (0.53-0.76) (log-odds ratio) et al, 2015°° 

Risk of recurrence, major depression: 0.942 Normal Amick et al, 2015'%° 
CBT plus pharmacotherapy vs (0.72-1.21) (log-odds ratio) 

pharmacotherapy only 

Risk of recurrence, major depression: 0.78° Normal Churchill et al, 20019” 
Individual CBT vs group CBT (0.69-0.93) (log-odds ratio) 

Risk of recurrence, major depression: 0.67 Normal Andrews et al, 20187! 
Guided iCBT vs usual care (0.54—0.83) (log-odds ratio) 

Risk of recurrence, second-generation 0.80 Normal Cipriani et al, 200915! 
antidepressant, sertraline (0.69-0.93) (log-odds ratio) 


Abbreviations: CBT, cognitive behavioural therapy, relates to face-to-face CBT; Cl, confidence interval; iCBT, internet-delivered CBT. 
4Recalculated from the reported relative risk of 1.06 (95% Cl 0.82-1.38). 


Relative risk estimated from the odds ratio after accounting for a mean baseline probability of major depressive episode in the placebo 
arm of 0.57.'52 


Ontario Health Technology Assessment Series; Vol. 19: No. 6, pp. 1-199, February 2019 175 


Appendices 


Table A9: Health State Utilities Used in the Economic Model 


February 2019 


Mean (SD/SE, 
Model Parameter 95% Cl) Distribution® Source 
Utilities 
Acute phase, untreated new major Beta Mohiuddin and Payne, 
depression 2014153 
Mild MDE 0.69 (SD 0.14) 
Moderate to severe MDE 0.52 (SD 0.28) 
Past major depression Beta Schaffer et al, 200210 
Mild MDE 0.79 (SD 0.28) 
Moderate to severe MDE 0.67 (SD 0.36) 
Guided iCBT Beta Hedman et al, 201 158.104 
4 months 0.82 (SD 0.18) 
12 months 0.85 (SD 0.14) 
Face-to-face CBT Fixed King et al, 20001 
4 months 0.85 (NR) 
12 months 0.85 (NR) 
Pharmacotherapy only—treated major 0.63 (SD 0.19) Beta Revicki and Wood, 
depression 1998154 
Normal health, well health state 0.94 (SE 0.03) Beta Lenert et al, 2000! 
Complex depression state, with 0.52 (SD 0.29) Beta Mann et al, 2009,'5° 
treatment, nonhospitalized Vallejo-Torres et al, 
2015156 
Complex depression state, severe 0.34 (SE 0.02) Beta Mann et al, 2009,'5° 


phase, hospitalized 


Vallejo-Torres et al, 
2015156 


Abbreviations: CBT, cognitive behavioural therapy; Cl, confidence interval; MDE, major depressive episode; NR, not reported; 

SD, standard deviation, SE, standard error. 

Assumed to be the same for anxiety disorders and major depression, based on results of a study in patients with social anxiety disorder. 
Beta distributions assigned in probabilistic sensitivity analysis (see above formulas for the calculations of the beta distribution’s parameters: 
a and B. 
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Table A10: Costs Used in the Economic Model’: Interventions, Usual Care, Follow-Ups, 
Complex Depression, and Hospitalization 


Mean, $ 
ORCI G (SE) Distribution Source 
Costs of providing iCBT and CBT 
Initial assessment, one-time costs, total: 223.35 Gamma? 
(55.84) 
GP referral 62.75 OHIP code K005"'° 
Assessment by psychiatrist or 80.30 OHIP code K1971'° 
psychologist 
CBT provided by nonphysician, weekly° Gamma Expert consultation, 
literature '5”-1®: annual 
salary of $130,000 plus 
30% benefits '®°° 
Individual session (1 hr/wk) 101.93 
(25.48) 
Group session (2 hr/wk) 20.93 
(5.10) 
Therapist-guided iCBT, one-time cost 1,466 Gamma Table 13 
Coach-guided iCBT, one-time cost4 1,389 
Employer-led iCBT, one-time cost? 887 
Costs of pharmacotherapy,° per week 
Sertraline, 50 mg/d 6.85 Fixed ODB"''; Kolber et al, 
2014112 
Sertraline, 100 mg/d 7.44 Fixed ODB"''; Kolber et al, 
2014112 
Venlafaxine, 75 mg/d 5.93 Fixed ODB""'; Kolber et al, 
2014112 
Costs of follow-up, per week 
CBT only (mild major depression) 20.85 Gamma Calculated based on 
(5.21) OHIP codes K005 and 
K198"'9: details provided 
in text 
CBT plus pharmacotherapy (moderate to 33.20 Gamma Calculated based on 
severe major depression) (8.30) OHIP codes K005 and 
K198"'9: details provided 
in text 
Pharmacotherapy, usual care (moderate 33.20 Gamma Calculated based on 
to severe major depression) (8.30) OHIP codes K005 and 
K198'': details provided 
in text 
No treatment, usual care (mild major 7.24 Gamma Calculated based on 
depression) (1.71) OHIP codes K005 and 
K198'"°: details provided 
in text 
Costs: Complex depression 
Responders, weekly 163.78 Gamma Health Quality Ontario, 
(40.92) 201616! 
Nonresponders, weekly® 222.47 Gamma Health Quality Ontario, 
(55.61) 201616! 


Ontario Health Technology Assessment Series; Vol. 19: No. 6, pp. 1-199, February 2019 177 


Appendices February 2019 


Mean, $ 
FIR ETOG Eu (SE) Distribution Source 
One-time costs of hospitalization 5,422.88 Gamma Bereza et al, 201216 
(per 10 days) (1,355.72) 
Costs: Well health state, per week 
Follow-up by GP (2 visits per year) 2.41 Gamma Calculated based on 
(0.60) OHIP code K005"'° 


Abbreviations: CBT, cognitive behavioural therapy, relates to face-to-face CBT; iCBT, internet-delivered CBT; GP, general practitioner; 
ODB, Ontario Drug Benefit program; OHIP, Ontario Health Insurance Plan Schedule of Benefits and Fees; SE, standard error. 
4Original nondiscounted costs in 2017 Canadian dollars, estimated per weekly cycle; standard error based on an assumption that the 
mean costs vary by +25%.Two parameters of the gamma distribution (a,A) are derived from the mean and SE. Formulas for these 
calculations are: a = (Mean?)/(SE*); A= Mean/([MeanxSE}’). 

‘Used in probabilistic sensitivity scenario analyses. 

°Labour costs after applied salary adjustment for a full-time professional working 1,658 hours per year. 

4 Used in scenario analyses 

Costs of drugs include a dispensing fee of $10.22 and a mark-up of $1.71 for 30-day use.'!2 
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Appendix 7: Scenario Analysis: Lower Costs of Face-to-Face Delivery Compared 
With iCBT 


Table A11: Cost-Utility Analysis, Sequential Approach: Internet-Delivered CBT, In-Person CBT, 
and Usual Care for Major Depression, Lower Delivery Costs of In-Person CBT 


Mean Costs? Incremental Incremental ICER 
Strategy ($) Mean QALYs Costs*($) QALYs° $/QALY gained 
Unguided iCBT 274.62 0.778 - - 
Usual care 409.40 0.789 - - - 
Group CBT 1,582.54 0.817 Dominated 
Guided iCBT 1,668.91 0.827 1,259 0.04 39,661 
Individual CBT 2,467.08 0.827 - - Dominated 


Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; ICER, incremental cost-effectiveness ratio; QALY, quality- 
adjusted life-year. 

All costs in 2018 Canadian dollars. No discounting was done because of the 12-month time horizon. 

Incremental cost = average cost (guided CBT) — average cost (usual care). 

°Incremental effect = average effect (guided CBT) — average effect (usual care). 

Note: Results may appear incorrect due to rounding. 


Table A12: Cost—-Utility Analysis, Sequential Approach: Internet-Delivered CBT, In-Person CBT, 
and Usual Care for Anxiety Disorders, Lower Delivery Costs of In-Person CBT 


Mean Costs? Mean Incremental Incremental ICER 

Strategy ($) QALYs Costs? ($) QALYs° $/QALY gained 

Unguided iCBT 269.69 0.801 = = 

Usual care 409.40 0.787 - - Dominated 

Group CBT 866.52 0.817 596.8 0.015 38,515 

Individual CBT 1495.53 0.827 - - Extended 
Dominance 

Guided CBT 1666.59 0.833 800.1 0.017 47,753 


Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; ICER, incremental cost-effectiveness ratio; QALY, quality- 
adjusted life-year. 

All costs in 2018 Canadian dollars. No discounting was done because of the 12-month time horizon. 

Incremental cost = average cost (guided CBT) — average cost (unguided iCBT). 

‘Incremental effect = average effect (guided CBT) - average effect (unguided iCBT). 

Note: Results may appear incorrect due to rounding. 
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Appendix 8: Scenario Analysis: Medication Starting From Baseline 


Table A13: Cost—-Utility Analysis, Sequential Approach: Internet-Delivered CBT, In-Person CBT, 
Usual Care for Major Depression, and Medication From Beginning of Intervention 


Mean Costs? Mean Incremental Incremental ICER, 
Strategy ($) QALYs Costs? ($) QALYs° $/QALY gained 
Unguided iCBT 794.22 0.778 - - 
Usual care 862.23 0.789 - - 
Group CBT 2,132.44 0.817 s = Extended 

Dominance 

Guided iCBT 2,188.82 0.827 1,328 0.04 33,397 
Individual CBT 3,096.02 0.827 907 0.0001 9,216,211 


Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; ICER, incremental cost-effectiveness ratio; QALY, quality- 
adjusted life-year. 

All costs in 2018 Canadian dollars. No discounting was done because of the 12-month time horizon. 

Incremental cost = average cost (guided CBT) — average cost (usual care). 

‘Incremental effect = average effect (guided CBT) — average effect (usual care). 

Note: Results may appear incorrect due to rounding. 


Table A14: Cost—-Utility Analysis, Sequential Approach: Internet-Delivered CBT, In-Person CBT, 
Usual Care for Anxiety Disorders, and Medication From Beginning of Intervention 


Mean Costs? Mean Incremental Incremental ICER 

Strategy ($) QALYs Costs? ($) QALYs° $/QALY gained 

Unguided iCBT 790.43 0.801 - - 

Usual care 862.26 0.788 - - Dominated 

Group CBT 1,719.15 0.817 - - Extended 
Dominance 

Guided iCBT 2,188.54 0.833 1,398 0.03 43,250 

Individual CBT 2,398.23 0.827 - - Dominated 


Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; ICER, incremental cost-effectiveness ratio; QALY, quality- 
adjusted life-year. 

4A ll costs in 2018 Canadian dollars. No discounting was done because of the 12-month time horizon. 

Incremental cost = average cost (guided CBT) — average cost (unguided iCBT). 

‘Incremental effect = average effect (guided CBT) — average effect (unguided iCBT). 

Note: Results may appear incorrect due to rounding. 
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Appendix 9: Results: Life-Time Cost-Effectiveness Analysis of Guided iCBT 
Followed by In-Person CBT Versus Usual Care 


In a sequential cost-utility analysis ranking three strategies by increasing costs, usual care was 
dominated (the highest costs and lowest benefits) and excluded from the final calculations 
(Table A15). Therefore, as compared to usual care, both types of guided CBT— internet and in- 
person—strategies were cost saving and associated with increments in QALYs. Over a lifetime 
time horizon using a stepped-care model, guided iCBT by a therapist followed by individual 
face-to-face CBT represented an economically attractive option when compared with guided 
iCBT followed by group CBT. It was associated with increases in both the mean discounted 
quality-adjusted survival of 0.086 QALYs (95% Crl -0.11 to 0.20) and the mean discounted 
costs of $39.82 (95% Crl -$2,886 to $2,351), yielding an ICER of $1,098 per QALY gained. 


Table A15: Lifetime Cost-Effectiveness of Guided iCBT Followed by Face-To-Face CBT, Cosi- 
Utility Analysis With Sequential Approach: Stepped-Care Model 


Mean Incremental Incremental ICER, $/QALY 
Strategy Mean Costs, $2 QALYs Costs, $2" QALYs° Gained? 
Guided iCBT 280,498.29 18.300 
followed by group 
CBT 
Guided iCBT 280,538.11 18.336 39.82 0.036 1,098 
followed by 
individual CBT 
Usual care 283,651.49 18.090 3,113.37 -0.246 Dominated 


Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; ICER, incremental cost-effectiveness ratio; QALY, quality- 


adjusted life-year. 
All costs in 2018 Canadian dollars. Costs and QALYs discounted at 1.5% rate. 
Incremental cost = average cost (strategy of guided CBT followed by individual CBT) — average cost (strategy of guided CBT followed by group 


CBT). 
‘Incremental effect = average effect (strategy of guided CBT followed by individual CBT) — average effect (strategy of guided CBT followed by 


group CBT). 
Note: Results may appear incorrect because of rounding. 


In this cost—utility analysis considering a stepped-care modeling of two different strategies of 
internet-delivered and in-person CBT over a lifetime time horizon, the probability of cost- 
effectiveness of guided iCBT followed by individual face-to-face CBT was 60% at a willingness- 
to-pay value of $50,000 per QALY gained, increasing to 63% at $100,000 per QALY gained 
(Figure 8). 
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Figure A2: Cost-Effectiveness Acceptability Curve: Guided iCBT in a Stepped-Care Model 
Including In-Person CBT (Individual or Group) 

Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; QALY, quality-adjusted life-year; WTP, willingness to pay. 

Note: The cost-effectiveness acceptability curve graphically presents the probability of cost-effectiveness of the examined CBT strategies and usual 


care across various willingness-to-pay values on the x—y coordinate system. The x-axis shows the probability of cost-effectiveness (range: 0-1) and 
the y-axis represents willingness-to-pay values (range: $0 to $100,000 per QALY gained). 


Sensitivity Scenario Analyses: Lifetime Cost-Effectiveness of Guided iCBT Within 
a Stepped-Care Model 


We examined parameter and structural model uncertainty in scenario analyses (Table A16). 
The results remained robust in all scenarios with guided iCBT followed by individual CBT as the 
most economically attractive strategy. The probability of cost-effectiveness of this strategy ata 
willingness-to-pay value of $100,000 per QALY gained ranged from 63% to 65% in scenarios 
addressing parameter uncertainty (e.g., changes in the efficacy of iCBT and CBT, dropout rates, 
utilities, and medication costs). The probability of cost-effectiveness of guided iCBT followed by 
individual CBT increased from 60% (reference case) to 99% (5-yr time horizon). 
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Table A16: Sensitivity Scenario Analysis Results: iCBT Followed by In-Person CBT ina 


Stepped-Care Model? 


Guided iCBT With 
Individual CBT vs. 


Guided iCBT With 


Guided iCBT With 


Guided iCBT With Individual CBT vs. Group CBT vs. 
Group CBT, Usual Care, Usual Care, 

Parameter/Assumption: ICER ($/QALY) ICER ($/QALY) ICER ($/QALY) 
1. Efficacy of CBT: decreased by 25% 
a. Base case 1,098 Cost saving Cost saving 
b. Scenario: 0.25 x RR_iCBT & RR_CBT Dominant Cost saving Cost saving 
2. Dropout: increased 2 times in CBT 
a. Base case: 19% (iCBT) and 16% (CBT) 1,098 Cost saving Cost saving 
b. Scenarios: 2 x base case probability Dominant 846 Dominated 
3. Utility of health states in which 
iCBT/CBT was provided: 10% lower 
a. Base case: 0.85 at 12 months 1,098 Cost saving Cost saving 
b. Scenarios: 10% lower 560 Cost saving Cost saving 
4. Medication costs 
a. Base case 1,098 Cost saving Cost saving 
b. Scenario: Increased 2x (all strategies) 1,097 Cost saving Cost saving 
5. Time horizon 
a. Base case: lifetime 1,098 Cost saving Cost saving 
b. Scenario: 5 years 6,088 Cost saving Cost saving 


Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered cognitive behavioural therapy; ICER, incremental cost-effectiveness 


ratio; QALY, quality-adjusted life-year; RR, relative risk. 
4All costs in 2018 Canadian dollars, discounted at 1.5%. 
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Appendix 10: Letter of Information 


CALL FOR PARTICIPATION FROM HEALTH QUALITY ONTARIO 


REVIEW OF 
INTERNET COGNITIVE BEHAVIOURAL THERAPY (iCBT) 


Have you or someone you are caring for experienced anxiety or depression? If you or your loved one 
received or is considering internet-based CBT, we'd like to hear about your experience. 


Interviews are taking place till the end of May 2018. It takes 20 to 30 minutes of your time and can be 


done either on the phone or in-person. 


WHY GET INVOLVED? 


Your participation will help Health Quality Ontario in its review of the usefulness of internet based 
Cognitive Behavioural Therapy. This review will result in a recommendation of whether this therapy 
should be publicly funded. 


ABOUT US 


Health Quality Ontario is a provincial agency with one purpose: better health for all Ontarians. Part of 


our work involves conducting reviews of various health care technologies and services to gauge their 
usefulness. 


if you're interested in participating, or have questions, please contact: 
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Appendix 11: Interview Guide 


Background 


Provide information on Health Quality Ontario’s mandate. 

Explain the health technology assessment program and part of Patient, Caregiver and 
Public Engagement. Explain the purpose of the interview. 

Confirm consent for audio-recording 

Restate options of withdrawal, freedom of sharing and not-sharing of information 


Lived Experience 


What are the biggest challenges of living/caring for someone with anxiety or depression? 
How does it impact your day to day routine? How would you describe your quality of life? 


Therapies 


What are the current therapies/treatments that you aware of? 

What therapies/treatments are accessible to you? Did you face any barriers? 

Which therapies/treatments have you explored? And why did you explore these? 

How did the therapies/treatments meet your needs? 

How did the therapies impact your quality of life? 

What were the side-effects and benefits? 

Were there any equity issues related to cost, access, knowledge of health care system? 


Internet-Delivered Cognitive Behavioural Therapy 


Please explain the process of therapy 

How did this therapy meet/not meet your needs? How was it adequate/inadequate? 
Quality of life’? Empowerment? Ownership? Adherence? Lifestyle? 

What were the side-effects and benefits? Anxiety, painful, intrusiveness? 

Were there equity issues related to cost, access, knowledge of health care system, etc.? 
Travel, repeat visits? 

What challenges did this procedure address? 


Ontario Health Technology Assessment Series; Vol. 19: No. 6, pp. 1-199, February 2019 185 


February 2019 


REFERENCES 


(1) 


Health Quality Ontario. Major depression: care for adults and adolescents [Internet]. 
Toronto: Queen's Printer for Ontario; 2016 [cited 2018 Aug 18]. Available from: 
http:/Awww.hgontario.ca/portals/0/documents/evidence/quality-standards/qs-depression- 


clinical-quide-1609-en.paf 
Patten SB, Kennedy SH, Lam RW, O'Donovan C, Filteau Mu, Parikh SV, et al. Canadian 


Network for Mood and Anxiety Treatments (CANMAT) clinical guidelines for the 
management of major depressive disorder in adults. |. Classification, burden and 
principles of management. J Affect Disord. 2009;117 Suppl 1:S5-S14. 

American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 
5th ed. Washington (DC): The Association; 2013. 

Kennedy SH, Lam RW, Parikh SV, Patten SB, Ravindran AV. Canadian Network for 
Mood and Anxiety Treatments (CANMAT) clinical guidelines for the management of 
major depressive disorder in adults. Introduction. J Affect Disord. 2009;117 Suppl 1:S1- 
2. 

Lam RW, Kennedy SH, Grigoriadis S, McIntyre RS, Milev R, Ramasubbu R, et al. 
Canadian Network for Mood and Anxiety Treatments (CANMAT) clinical guidelines for 
the management of major depressive disorder in adults. III. Pharmacotherapy. J Affect 
Disord. 2009;117 Suppl! 1:S26-43. 

Parikh SV, Segal ZV, Grigoriadis S, Ravindran AV, Kennedy SH, Lam RW, et al. 
Canadian Network for Mood and Anxiety Treatments (CANMAT) clinical guidelines for 
the management of major depressive disorder in adults. Il. Psychotherapy alone or in 
combination with antidepressant medication. J Affect Disord. 2009;117 Suppl 1:S15-25. 
Adelman CB, Panza KE, Bartley CA, Bontempo A, Bloch MH. A meta-analysis of 
computerized cognitive-behavioral therapy for the treatment of DSM-5 anxiety disorders. 
J Clin Psychiatry. 2014;75(7):e695-704. 

Kupfer DJ. Anxiety and DSM-5. Dialogues Clin Neurosci. 2015;17(3):245-6. 

National Collaborating Centre for Mental Health. Generalised anxiety disorder in adults: 
management in primary, secondary and community care. NICE clinical guidelines, No. 
113 [Internet]. Leicester (UK): British Psychological Society; 2011 [cited 2016 Jun 5]. 
Available from: https://www.ncbi.nim.nih.gov/pubmedhealth/PMH0041616 

Pelletier L, O'Donnell S, McRae L, Grenier J. The burden of generalized anxiety disorder 
in Canada. Health Promot Chronic Dis Prev Can. 2017;37(2):54-62. 

Statistics Canada. Table 13-10-0465-01: Mental health indicators [Internet]. Ottawa 
(ON): Statistics Canada; 2018 [cited 2018 Oct 5]. Available from: 
httos:/Awww150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=1310046501 

Mood Disorders Society of Canada. Chapter 5 Anxiety Disorders. Belleville, Ontario: 
N.D. 

Churchill R, Moore TH, Furukawa TA, Caldwell DM, Davies P, Jones H, et al. 'Third 
wave' cognitive and behavioural therapies versus treatment as usual for depression. 
Cochrane Database Syst Rev. 2013;10:CD008705. 

Gratzer D, Khalid-Khan F. Internet-delivered cognitive behavioural therapy in the 
treatment of psychiatric illness. CMAJ. 2016;188(4):263-72. 

Olthuis JV, Watt MC, Bailey K, Hayden JA, Stewart SH. Therapist-supported internet 
cognitive behavioural therapy for anxiety disorders in adults. Cochrane Database Syst 
Rev. 2015;3:CD011565. 

Katzman MA, Bleau P, Blier P, Chokka P, Kjernisted K, Van Ameringen M, et al. 
Canadian clinical practice guidelines for the management of anxiety, posttraumatic 
stress and obsessive-compulsive disorders. BMC Psychiatry. 2014;14(1):S1. 


Ontario Health Technology Assessment Series; Vol. 19: No. 6, pp. 1-199, February 2019 186 


References February 2019 


(17) 


(18) 


(19) 


(20) 


(21) 


(22) 


(23) 


National Institute for Health and Care Excellence. Depression in adults: recognition and 
management [Internet]. London: The Institute; 2009 [cited 2018 Apr 6]. Available from: 
https://www.nice.org.uk/quidance/cg90 

McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer 
review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. 
2016;75:40-6. 

Whiting P, Savovié J, Higgins JP, Caldwell DM, Reeves BC, Shea B, et al. ROBIS: a 
new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol. 
2016;69:225-34. 

Arnberg FK, Linton SJ, Hultcrantz M, Heintz E, Jonsson U. Internet-delivered 
psychological treatments for mood and anxiety disorders: a systematic review of their 
efficacy, safety, and cost-effectiveness. PLoS One. 2014;9(5):e98118. 

Andrews G, Basu A, Cuijpers P, Craske MG, McEvoy P, English CL, et al. Computer 
therapy for the anxiety and depression disorders is effective, acceptable and practical 
health care: an updated meta-analysis. J Anxiety Disord. 2018;55:70-8. 

Kaltenthaler E, Parry G, Beverley C, Ferriter M. Computerised cognitive-behavioural 
therapy for depression: systematic review. Br J Psychiatry. 2008;193(3):181-4. 

Dedert E, McDuffie JR, Swinkels C, Shaw R, Fulton J, Allen KD, et al. Computerized 
cognitive behavioral therapy for adults with depressive or anxiety disorders [Internet]. 
Washington (DC): U.S. Department of Veterans Affairs; 2013 [cited 2018 Oct 11]. 
Available from: https://www.ncbi.nim.nih.gov/books/NBK269001/ 

Richards D, Richardson T, Timulak L, McElvaney J. The efficacy of internet-delivered 
treatment for generalized anxiety disorder: a systematic review and meta-analysis. 
Internet Interv. 2015;2(3):272-82. 

Kampmann IL, Emmelkamp PM, Morina N. Meta-analysis of technology-assisted 
interventions for social anxiety disorder. J Anxiety Disord. 2016;42:71-84. 

Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for 
systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 
2009;6(6):e1000097. 

Young C, Helis E, Williams D. Internet-delivered cognitive behavioural therapy for major 
depressive disorder and anxiety disorders: an environmental scan (environmental scan 
no. 78) [Internet]. Ottawa (ON)2019. 

Ahern E, Kinsella S, Semkovska M. Clinical efficacy and economic evaluation of online 
cognitive behavioral therapy for major depressive disorder: a systematic review and 
meta-analysis. Expert Rev Pharmacoecon Outcomes Res. 2018;18(1):25-41. 

Paganini S, Teigelkotter W, Buntrock C, Baumeister H. Economic evaluations of 
internet- and mobile-based interventions for the treatment and prevention of depression: 
a systematic review. J Affect Disord. 2018;225:733-55. 

Kolovos S, van Dongen JM, Riper H, Buntrock C, Cuijpers P, Ebert DD, et al. Cost 
effectiveness of guided internet-based interventions for depression in comparison with 
control conditions: an individual-participant data meta-analysis. Depress Anxiety. 
2018;35(3):209-19. 

Brettschneider C, Djadran H, Harter M, Lowe B, Riedel-Heller S, Konig HH. Cost-utility 
analyses of cognitive-behavioural therapy of depression: a systematic review. 
Psychother Psychosom. 2015;84(1):6-21. 

Ophuis RH, Lokkerbol J, Heemskerk SC, van Balkom Au, Hiligsmann M, Evers SM. 
Cost-effectiveness of interventions for treating anxiety disorders: a systematic review. J 
Affect Disord. 2017;210:1-13. 

Lewis C, Pearce J, Bisson JI. Efficacy, cost-effectiveness and acceptability of self-help 
interventions for anxiety disorders: systematic review. Br J Psychiatry. 2002;200(1):15- 
21. 


Ontario Health Technology Assessment Series; Vol. 19: No. 6, pp. 1-199, February 2019 187 


References February 2019 


(34) 


(35) 


(36) 


(37) 


(38) 


Donker T, Blankers M, Hedman E, Ljotsson B, Petrie K, Christensen H. Economic 
evaluations of internet interventions for mental health: a systematic review. Psychol 
Med. 2015;45(16):3357-76. 

Hedman E, Ljotsson B, Lindefors N. Cognitive behavior therapy via the internet: a 
systematic review of applications, clinical efficacy and cost-effectiveness. Expert Rev 
Pharmacoecon Outcomes Res. 2012;12(6):745-64. 

Littlewood E, Duarte A, Hewitt C, Knowles S, Palmer S, Walker S, et al. A randomised 
controlled trial of computerised cognitive behaviour therapy for the treatment of 
depression in primary care: the Randomised Evaluation of the Effectiveness and 
Acceptability of Computerised Therapy (REEACT) trial. Health Technol Assess. 
2015;19(101):viii, xxi-171. 

Duarte A, Walker S, Littlewood E, Brabyn S, Hewitt C, Gilbody S, et al. Cost- 
effectiveness of computerized cognitive-behavioural therapy for the treatment of 
depression in primary care: findings from the Randomised Evaluation of the 
Effectiveness and Acceptability of Computerised Therapy (REEACT) trial. Psychol Med. 
2017;47(10):1825-35. 

Brabyn S, Araya R, Barkham M, Bower P, Cooper C, Duarte A, et al. The second 
Randomised Evaluation of the Effectiveness, cost-effectiveness and Acceptability of 
Computerised Therapy (REEACT-2) trial: does the provision of telephone support 
enhance the effectiveness of computer-delivered cognitive behaviour therapy? A 
randomised controlled trial. Health Technol Assess. 2016;20(89):1-64. 

Dixon P, Hollinghurst S, Edwards L, Thomas C, Foster A, Davies B, et al. Cost- 
effectiveness of telehealth for patients with depression: evidence from the healthlines 
randomised controlled trial. BUPsych Open. 2016;2(4):262-9. 

Hollinghurst S, Peters TJ, Kaur S, Wiles N, Lewis G, Kessler D. Cost-effectiveness of 
therapist-delivered online cognitive-behavioural therapy for depression: randomised 
controlled trial. Br J Psychiatry. 2010;197(4):297-304. 

Phillips R, Schneider J, Molosankwe |, Leese M, Foroushani PS, Grime P, et al. 
Randomized controlled trial of computerized cognitive behavioural therapy for 
depressive symptoms: effectiveness and costs of a workplace intervention. Psychol 
Med. 2014;44(4):741-52. 

McCrone P, Knapp M, Proudfoot J, Ryden C, Cavanagh K, Shapiro DA, et al. Cost- 
effectiveness of computerised cognitive-behavioural therapy for anxiety and depression 
in primary care: randomised controlled trial. Br J Psychiatry. 2004;185:55-62. 
Romero-Sanchiz P, Nogueira-Arjona R, Garcia-Ruiz A, Luciano JV, Garcia Campayo J, 
Gili M, et al. Economic evaluation of a guided and unguided internet-based CBT 
intervention for major depression: results from a multi-center, three-armed randomized 
controlled trial conducted in primary care. PLoS One. 2017;12(2):e01 72741. 

Titov N, Dear BF, Ali S, Zou JB, Lorian CN, Johnston L, et al. Clinical and cost- 
effectiveness of therapist-guided internet-delivered cognitive behavior therapy for older 
adults with symptoms of depression: a randomized controlled trial. Behav Ther. 
2015;46(2):193-205. 

Geraedts AS, van Dongen JM, Kleiboer AM, Wiezer NM, van Mechelen W, Cuijpers P, 
et al. Economic evaluation of a web-based guided self-help intervention for employees 
with depressive symptoms: results of a randomized controlled trial. J Occup Environ 
Med. 2015;57(6):666-75. 

Gerhards SA, de Graaf LE, Jacobs LE, Severens JL, Huibers Mu, Arntz A, et al. 
Economic evaluation of online computerised cognitive-behavioural therapy without 
support for depression in primary care: randomised trial. Br J Psychiatry. 
2010;196(4):310-8. 


Ontario Health Technology Assessment Series; Vol. 19: No. 6, pp. 1-199, February 2019 188 


References February 2019 


(47) 


(48) 


(49) 


(50) 


(51) 


(52) 


(54) 


(55) 


(56) 


Warmerdam L, van Straten A, Jongsma J, Twisk J, Cuijpers P. Online cognitive 
behavioral therapy and problem-solving therapy for depressive symptoms: exploring 
mechanisms of change. J Behav Ther Exp Psychiatry. 2010;41(1):64-70. 

Solomon D, Proudfoot J, Clarke J, Christensen H. e-CBT (myCompass), antidepressant 
medication, and face-to-face psychological treatment for depression in Australia: a cost- 
effectiveness comparison. J Med Internet Res. 2015;17(11):e255. 

Lee YC, Gao L, Dear BF, Titov N, Mihalopoulos C. The cost-effectiveness of the online 
MindSpot Clinic for the treatment of depression and anxiety in Australia. J Ment Health 
Policy Econ. 2017;20(4):155-66. 

Kaltenthaler E, Brazier J, De Nigris E, Tumur |, Ferriter M, Beverley C, et al. 
Computerised cognitive behaviour therapy for depression and anxiety update: a 
systematic review and economic evaluation. Health Technol Assess. 2006;10(33):iii, xi- 
xiv, 1-168. 

Mihalopoulos C, Kiropoulos L, Shih ST, Gunn J, Blashki G, Meadows G. Exploratory 
economic analyses of two primary care mental health projects: implications for 
sustainability. Med J Aust. 2005;183(10 Suppl):S73-S6. 

McCrone P, Marks IM, Mataix-Cols D, Kenwright M, McDonough M. Computer-aided 
self-exposure therapy for phobia/panic disorder: a pilot economic evaluation. Cogn 
Behav Ther. 2009;38(2):91-9. 

Titov N, Andrews G, Johnston L, Schwencke G, Choi |. Shyness programme: longer 
term benefits, cost-effectiveness, and acceptability. Aust N Z J Psychiatry. 
2009;43(1):36-44. 

Bergstrom J, Andersson G, Ljotsson B, Ruck C, Andreewitch S, Karlsson A, et al. 
Internet-versus group-administered cognitive behaviour therapy for panic disorder in a 
psychiatric setting: a randomised trial. BMC Psychiatry. 2010;10:54. 

Joesch JM, Sherbourne CD, Sullivan G, Stein MB, Craske MG, Roy-Byrne P. 
Incremental benefits and cost of coordinated anxiety learning and management for 
anxiety treatment in primary care. Psychol Med. 2012;42(9):1937-48. 

Nordgren LB, Hedman E, Etienne J, Bodin J, Kadowaki A, Eriksson S, et al. 
Effectiveness and cost-effectiveness of individually tailored internet-delivered cognitive 
behavior therapy for anxiety disorders in a primary care population: a randomized 
controlled trial. Behav Res Ther. 2014;59:1-11. 

Hedman E, Andersson E, Lindefors N, Andersson G, Ruck C, Ljotsson B. Cost- 
effectiveness and long-term effectiveness of internet-based cognitive behaviour therapy 
for severe health anxiety. Psychol Med. 2013;43(2):363-74. 

Hedman E, Andersson E, Ljotsson B, Andersson G, Ruck C, Lindefors N. Cost- 
effectiveness of internet-based cognitive behavior therapy vs. cognitive behavioral group 
therapy for social anxiety disorder: results from a randomized controlled trial. Behav Res 
Ther. 2011 ;49(11):729-36. 

Hedman E, Axelsson E, Gorling A, Ritzman C, Ronnheden M, El Alaoui S, et al. Internet- 
delivered exposure-based cognitive-behavioural therapy and behavioural stress 
management for severe health anxiety: randomised controlled trial. Br J Psychiatry. 
2014;205(4):307-14. 

Hedman E, Andersson E, Ljotsson B, Axelsson E, Lekander M. Cost effectiveness of 
internet-based cognitive behaviour therapy and behavioural stress management for 
severe health anxiety. BMJ Open. 2016;6(4):e009327. 

El Alaoui S, Hedman-Lagerlof E, Ljotsson B, Lindefors N. Does internet-based cognitive 
behaviour therapy reduce healthcare costs and resource use in treatment of social 
anxiety disorder? A cost-minimisation analysis conducted alongside a randomised 
controlled trial. BMJ Open. 2017;7(9):e01 7053. 


Ontario Health Technology Assessment Series; Vol. 19: No. 6, pp. 1-199, February 2019 189 


References February 2019 


(62) 


(63) 


(64) 


Dear BF, Zou JB, Ali S, Lorian CN, Johnston L, Sheehan J, et al. Clinical and cost- 
effectiveness of therapist-guided internet-delivered cognitive behavior therapy for older 
adults with symptoms of anxiety: a randomized controlled trial. Behav Ther. 
2015;46(2):206-17. 

Kumar S, Jones Bell M, Juusola JL. Mobile and traditional cognitive behavioral therapy 
programs for generalized anxiety disorder: a cost-effectiveness analysis. PLoS One. 
2018;13(1):e0190554. 

National Collaborating Centre for Mental Health. Social anxiety disorder recognition, 
assessment and treatment. NICE clinical guidelines, No. 159 [Internet]. Leicester (UK): 
British Psychological Society; 2013 [cited 2016 Jun 05]. Available from: 
https://www.ncbi.nlm.nih.gov/oubmedhealth/PMH0071627 

Apolinario-Hagen J, Harrer M, Kahlke F, Fritsche L, Salewski C, Ebert DD. Public 
attitudes toward guided internet-based therapies: web-based survey study. JMIR Ment 
Health. 2018;5(2):e10735. 

Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. 
Consolidated Health Economic Evaluation Reporting Standards (CHEERS)— 
explanation and elaboration: a report of the ISPOR Health Economic Evaluation 
Publication Guidelines Good Reporting Practices Task Force. Value Health. 
2013;16(2):231-50. 

Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. 
Consolidated Health Economic Evaluation Reporting Standards (CHEERS)--explanation 
and elaboration: a report of the ISPOR Health Economic Evaluation Publication 
Guidelines Good Reporting Practices Task Force. Value Health. 2013;16(2):231-50. 
Canadian Agency for Drugs and Technologies in Health. Guidelines for the economic 
evaluation of health technologies: Canada, 4th edition [Internet]. Ottawa (ON): The 
Agency; 2017 [cited 2017 Apr 04]. Available from: https:/\www.cadth.ca/quidelines- 
economic-evaluation-health-technologies-canada-4th-edition 

Katz C, Stein MB, Sareen J. Anxiety disorders in the DSM-5: new rules on diagnosis and 
treatment [Internet]. Toronto (ON): Canadian Network for Mood and Anxiety 
Treatments; 2013 [cited 2018 Apr 13]. Available from: 
http://www.moodandanxietyrounds.ca/crus/144-010%20Enqlish.pdf 

Spitzer RL, Kroenke K, Williams JB. Validation and utility of a self-report version of 
PRIME-MD: the PHQ primary care study. Primary care evaluation of mental disorders. 
Patient health questionnaire. JAMA. 1999;282(18):1737-44. 

Lam RW, Kennedy SH, Parikh SV, MacQueen GM, Milev RV, Ravindran AV, et al. 
Canadian Network for Mood and Anxiety Treatments (CANMAT) 2016 clinical guidelines 
for the management of adults with major depressive disorder: introduction and methods. 
Can J Psychiatry. 2016;61(9):506-9. 

Kennedy SH, Lam RW, Mcintyre RS, Tourjman SV, Bhat V, Blier P, et al. Canadian 
Network for Mood and Anxiety Treatments (CANMAT) 2016 clinical guidelines for the 
management of adults with major depressive disorder: section 3. Pharmacological 
treatments. Can J Psychiatry. 2016;61 (9):540-60. 

Parikh SV, Quilty LC, Ravitz P, Rosenbluth M, Pavlova B, Grigoriadis S, et al. Canadian 
Network for Mood and Anxiety Treatments (CANMAT) 2016 clinical guidelines for the 
management of adults with major depressive disorder: section 2. Psychological 
treatments. Can J Psychiatry. 2016;61(9):524-39. 

Patten SB. Updated CANMAT guidelines for treatment of major depressive disorder. 
Can J Psychiatry. 2016;61(9):504-5. 

Kroenke K, Spitzer RL, Williams JBW. The PHQ-9: validity of a brief depression severity 
measure J Gen Intern Med. 2001;16:606-13. 


Ontario Health Technology Assessment Series; Vol. 19: No. 6, pp. 1-199, February 2019 190 


References February 2019 


(76) 


(77) 


(78) 


(79) 


(80) 


(81) 


(82) 


Shim RS, Baltrus P, Ye J, Rust G. Prevalence, treatment, and control of depressive 
symptoms in the United States: results from the National Health and Nutrition 
Examination Survey (NHANES), 2005-2008. J Am Board Fam Med. 201 1;24(1):33-8. 
Health Quality Ontario. Psychotherapy for major depressive disorder and generalized 
anxiety disorder: a health technology assessment. Ont Health Technol Assess Ser. 
2017;17(15):1-167. 

Chan SW, Adams M. Service use, drop-out rate and clinical outcomes: a comparison 
between high and low intensity treatments in an IAPT service. Behav Cogn Psychother. 
2014;42(6):747-59. 

Clark DM, Canvin L, Green J, Layard R, Pilling S, Janecka M. Transparency about the 
outcomes of mental health services (IAPT approach): an analysis of public data. Lancet. 
2018;391(10121):679-86. 

Clark DM. Implementing NICE guidelines for the psychological treatment of depression 
and anxiety disorders: the IAPT experience. Int Rev Psychiatry. 201 1;23(4):318-27. 
Cuijpers P, Donker T, van Straten A, Li J, Andersson G. Is guided self-help as effective 
as face-to-face psychotherapy for depression and anxiety disorders? A systematic 
review and meta-analysis of comparative outcome studies. Psychol Med. 
2010;40(12):1943-57. 

Andersson G, Cuijpers P, Carlbring P, Riper H, Hedman E. Guided internet-based vs. 
face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a 
systematic review and meta-analysis. World Psychiatry. 2014;13(3):288-95. 

Cuijpers P, van Straten A, Andersson G. Internet-administered cognitive behavior 
therapy for health problems: a systematic review. J Behav Med. 2008;31(2):169-77. 
Cuijpers P, van Straten A, Andersson G, van Oppen P. Psychotherapy for depression in 
adults: a meta-analysis of comparative outcome studies. J Consult Clin Psychol. 
2008;76(6):909-22. 

Johansson R, Andersson G. Internet-based psychological treatments for depression. 
Expert Rev Neurother. 2012;12(7):861-70. 

Clark DM. Overview of the UK Improving Access to Psychological Therapy (IAPT) 
program. Centre for Addiction and Mental Health; 2018. 

Richards D, Duffy D, Blackburn B, Earley C, Enrique A, Palacios J, et al. Digital IAPT: 
the effectiveness & cost-effectiveness of internet-delivered interventions for depression 
and anxiety disorders in the Improving Access to Psychological Therapies programme: 
study protocol for a randomised control trial. BMC Psychiatry. 2018;18(1):59. 

Canadian Psychiatric Association. Clinical practice guidelines. Management of anxiety 
disorders. Can J Psychiatry. 2006;51(8 Suppl):9S-91S. 

Excellence NIfHaC. Generalised anxiety disorder and panic disorder in adults: 
management [Internet]. London (UK): National Institute for Health and Care Excellence; 
2011 [cited 2018 Apr 07]. Available from: 

https://www.nice.org.uk/quidance/cqg 1 13/resources/generalised-anxiety-disorder-and- 
panic-disorder-in-adults-management-pdf-35109387756997 

Crome E, Shaw J, Baillie A. Costs and returns on training investment for empirically 
supported psychological interventions. Aust Health Rev. 2017;41(1):82-8. 

Karyotaki E, Riper H, Twisk J, Hoogendoorn A, Kleiboer A, Mira A, et al. Efficacy of self- 
guided internet-based cognitive behavioral therapy in the treatment of depressive 
symptoms: a meta-analysis of individual participant data. JAMA Psychiatry. 
2017;74(4):351-9. 

van Ballegooijen W, Cuijpers P, van Straten A, Karyotaki E, Andersson G, Smit JH, et al. 
Adherence to internet-based and face-to-face cognitive behavioural therapy for 
depression: a meta-analysis. PLoS One. 2014;9(7):e100674. 


Ontario Health Technology Assessment Series; Vol. 19: No. 6, pp. 1-199, February 2019 191 


References February 2019 


(93) 


(94) 


Koeser L, Donisi V, Goldberg DP, McCrone P. Modelling the cost-effectiveness of 
pharmacotherapy compared with cognitive-behavioural therapy and combination therapy 
for the treatment of moderate to severe depression in the UK. Psychol Med. 
2015;45(14):3019-31. 
Machado M, Iskedjian M, Ruiz |, Einarson TR. Remission, dropouts, and adverse drug 
reaction rates in major depressive disorder: a meta-analysis of head-to-head trials. Curr 
Med Res Opin. 2006;22(9):1825-37. 
Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction to meta-analysis. 1st 
ed. Chichester (UK): John Wiley & Sons Ltd.; 2009 Apr 27. 
Biesheuvel-Leliefeld KE, Kok GD, Bockting CL, Cuijpers P, Hollon SD, van Marwijk HW, 
et al. Effectiveness of psychological interventions in preventing recurrence of depressive 
disorder: meta-analysis and meta-regression. J Affect Disord. 2015;174:400-10. 
Churchill R, Hunot V, Corney R, Knapp M, McGuire H, Tylee A, et al. A systematic 
review of controlled trials of the effectiveness and cost-effectiveness of brief 
psychological treatments for depression. Health Technol Assess. 2001;5(35):1-173. 
Cipriani A, Geddes JR. Placebo for depression: we need to improve the quality of 
scientific information but also reject too simplistic approaches or ideological nihilism. 
BMC Med. 2014;12:105. 
Drummond MF, Sculpher MJT, G.W., O'Brien BJ, Stoddart GL. Methods for the 
economic evaluation of health care programmes. 3rd ed. New York: Oxford University 
Press 2005. 
Glanville J, Arber M, Veale T, Garcia S. Sensitivity of a search filter designed to identify 
studies reporting health state utility values. 116th Annual Meeting, Medical Library 
Association, Inc; 2016 May 15-20; Toronto, ON: Journal of the Medical Library 
Association. 
Glick HA, Doschi JA, Sonnad SS. Economic evaluations in clinical trials 2nd ed. Oxford 
(UK): Oxford University Press; 2007. 
Schaffer A, Levitt AJ, Hershkop SK, Oh P, MacDonald C, Lanctot K. Utility scores of 
symptom profiles in major depression. Psychiatry Res. 2002;110(2):189-97. 
King M, Sibbald B, Ward E, Bower P, Lloyd M, Gabbay M, et al. Randomised controlled 
trial of non-directive counselling, cognitive-behaviour therapy and usual general 
practitioner care in the management of depression as well as mixed anxiety and 
depression in primary care. Health Technol Assess. 2000;4(19):1-83. 
Hedman E, Andersson G, Andersson E, Ljotsson B, Ruck C, Asmundson Gu, et al. 
Internet-based cognitive-behavioural therapy for severe health anxiety: randomised 
controlled trial. Br J Psychiatry. 201 1;198(3):230-6. 
Lenert LA, Sherbourne CD, Sugar C, Wells KB. Estimation of utilities for the effects of 
depression from the SF-12. Med Care. 2000;38(7):763-70. 
Gidwani RA. A budget impact analysis of chronic disease screening: the business care 
of rapid HIV testing in VA emergency departments. Ann Arbor (MI): UMI Dissertation 
Publishing; 2011. 160 p. 
Wright JH, Wright AS, Albano AM, Basco MR, Goldsmith Lu, Raffield T, et al. Computer- 
assisted cognitive therapy for depression: maintaining efficacy while reducing therapist 
time. Am J Psychiatry. 2005;162(6):1158-64. 
Cornish P. Innovating online with stepped care: a research-based stepped care mental 
health community of practice proposal [Internet]. Palo Alto (CA): Issuu Inc.; 2014 [cited 
2018 Apr 6]. Available from: https://issuu.com/cacuss/docs/communique fall 2014 - 
web/9 
Sutherland G, Dinh T. Understanding the gap: a pan-Canadian analysis of prescription 
drug insurance coverage [Internet]. Ottawa (ON): Conference Board of Canada; 2017 


Ontario Health Technology Assessment Series; Vol. 19: No. 6, pp. 1-199, February 2019 192 


References February 2019 


(110) 


(111) 


(112) 


(113) 


(114) 


(115) 


(116) 


(117) 


(118) 


(119) 


(120) 


(121) 


(122) 


(123) 


(124) 


(125) 


[cited 2018 Apr 6]. Available from: http://innovativemedicines.ca/wp- 
content/uploads/2017/12/20170712-understanding-the-gap.padf 

Ministry of Health and Long-Term Care. Schedule of benefits: physician services under 
the Health Insurance Act [Internet]. Toronto (ON): The Ministry; 2015 [cited 2017 Oct 
11]. Available from: 

http://www. health.gov.on.ca/en/pro/programs/ohip/sob/physserv/sob_master20160401.p 
df 

Titov N, Andrews G, Davies M, Mcintyre K, Robinson E, Solley K. Internet treatment for 
depression: a randomized controlled trial comparing clinician vs. technician assistance. 
PLoS One. 2010;5(6):e10939. 

Kolber MC, Lee J, Nickonchuk T. Price comparison of commonly prescribed 
pharmaceuticals in Alberta 2014. [Internet]. Edmonton (AB): Alberta College of Family 
Physicians; 2014 [cited 2017 Apr 4]. Available from: https://www.acfp.ca/wp- 
content/uploads/2014/06/ACFPPricingDoc2014.pdf 

Statistics Canada. Table 14-10-0320-02: Average usual hours and wages by selected 
characteristics, monthly, unadjusted for seasonality (x 1,000) [Internet]. Ottawa (ON): 
Statistics Canada; 2018 [cited 2018 Oct 2]. Available from: 
https:/Awww150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=14 10032002 

Ministry of Labour. Minimum wage [Internet]. Toronto (ON): Queen's Printer for Ontario; 
2017 [cited 2018 Oct 2]. Available from: https://www.ontario.ca/document/your-quide- 
employment-standards-act-0/minimum-wage#section-0 

Clark DM. Realizing the mass public benefit of evidence-based psychological therapies: 
the IAPT program. Annu Rev Clin Psychol. 2018;14:159-83. 

Serhal E, Crawford A, Cheng J, Kurdyak P. Implementation and utilisation of 
telepsychiatry in Ontario: a population-based study. Can J Psychiatry. 2017;62(10):716- 
25 


Ahmad F, Shakya Y, Ginsburg L, Lou W, Ng PT, Rashid M, et al. Burden of common 
mental disorders in a community health centre sample. Can Fam Physician. 
2016;62(12):e758-e66. 

Alavi N, Hirji A, Sutton C, Naeem F. Online CBT is effective in overcoming cultural and 
language barriers in patients with depression. J Psychiatr Pract. 2016;22(1):2-8. 
Sullivan SD, Mauskopf JA, Augustovski F, Jaime Caro J, Lee KM, Minchin M, et al. 
Budget impact analysis-principles of good practice: report of the ISPOR 2012 Budget 
Impact Analysis Good Practice II Task Force. Value Health. 2014;17(1):5-14. 

Patten SB, Lee RC. Describing the longitudinal course of major depression using 
markov models: data integration across three national surveys. Popul Health Metr. 
2005;3:11. 

Patten SB, Williams JV, Lavorato DH, Fiest KM, Bulloch AG, Wang J. The prevalence of 
major depression is not changing. Can J Psychiatry. 2015;60(1):31-4. 

Patten SB, Williams JV, Lavorato DH, Wang JL, McDonald K, Bulloch AG. Descriptive 
epidemiology of major depressive disorder in Canada in 2012. Can J Psychiatry. 
2015;60(1):23-30. 

Patten SB, Williams JV, Lavorato DH, Wang JL, McDonald K, Bulloch AG. Major 
depression in Canada: what has changed over the past 10 years? Can J Psychiatry. 
2016;61(2):80-5. 

Tempier R, Meadows GN, Vasiliadis HM, Mosier KE, Lesage A, Stiller A, et al. Mental 
disorders and mental health care in Canada and Australia: comparative epidemiological 
findings. Soc Psychiatry Psychiatr Epidemiol. 2009;44(1):63-72. 

Morgan VA, Leonard H, Bourke J, Jablensky A. Intellectual disability co-occurring with 
schizophrenia and other psychiatric illness: population-based study. Br J Psychiatry. 
2008;193(5):364-72. 


Ontario Health Technology Assessment Series; Vol. 19: No. 6, pp. 1-199, February 2019 193 


References February 2019 


(126) 


(127) 


(128) 


(129) 


(130) 


(131) 


(132) 


(133) 


(134) 
(135) 


(136) 


(137) 


(138) 


(139) 


(140) 


Wozney L, McGrath P, Newton A, Hartling L, Curran J, Huguet A, et al. RE-AlMing e- 
mental health: a rapid review of current research [Internet]. Ottawa (ON): Mental Health 
Commision of Canada; 2017 [cited 2018 Apr 4]. Available from: 


https://www.mentalhealthcommission.ca/sites/default/files/201 7- 


08/eMH%20Literature%20Review FINAL%20EN.paf 
Eaton WW, Shao H, Nestadt G, Lee HB, Bienvenu OJ, Zandi P. Population-based study 


of first onset and chronicity in major depressive disorder. Arch Gen Psychiatry. 
2008;65(5):513-20. 

Casey L, Wright M-A, Clough BA. Comparison of perceived barriers and treatment 
preferences associated with internet-based and face-to-face psychological treatment of 
depression. Int J Cyber Behav Psychol Learning. 2014;4(4):16-22. 

Khatri N, Marziali E, Tchernikov |, Shepherd N. Comparing telehealth-based and clinic- 
based group cognitive behavioral therapy for adults with depression and anxiety: a pilot 
study. Clin Interv Aging. 2014;9:765-70. 

Williams C, McClay CA, Martinez R, Morrison J, Haig C, Jones R, et al. Online CBT life 
skills programme for low mood and anxiety: study protocol for a pilot randomized 
controlled trial. Trials. 2016;17(1):220. 

Barham L. Public and patient involvement at the UK National Institute for Health and 
Clinical Excellence. Patient. 2011;4(1):1-10. 

Messina J, Grainger DL. A pilot study to identify areas for further improvements in 
patient and public involvement in health technology assessments for medicines. Patient. 
2012;5(3):199-211. 

Ontario Health Technology Advisory Committee Public Engagement Subcommittee. 
Public engagement for health technology assessment at Health Quality Ontario—final 
report from the Ontario Health Technology Advisory Committee Public Engagement 
Subcommittee [Internet]. Toronto (ON): Queen's Printer for Ontario; 2015 Apr [cited 
2018 Apr 30]. Available from: 
http://www.hgontario.ca/Portals/0/documents/evidence/special-reports/report- 


subcommittee-20150407-en.pdf 
Kvale S. Interviews: an introduction to qualitative research interviewing. Thousand Oaks 


(CA): Sage; 1996. 

Kuzel AJ. Sampling in qualitative inquiry. In: Miller WL, Crabtree BF, editors. Doing 
qualitative research. Thousand Oaks (CA): Sage; 1999. p. 33-45. 

Health Technology Assessment International Interest Group on Patient and Citizen 
Involvement in HTA. Introduction to health technology assessment [Internet]. Edmonton 
(AB): Health Technology Assessment International; 2015 [cited 2016 Jan]. Available 
from: 


http:/Avww.htai.org/fileadmin/HTAi_ Files/ISG/PatientInvolvement/v2_files/Resource/PCl 


SG-Resource-Intro to HTA KFacey Jun13.paf 
Strauss AL, Corbin JM. Grounded theory research: procedures, canons, and evaluative 


criteria. Qual Sociol. 1990;13(1):3-21. 

Strauss AL, Corbin JM. Grounded theory methodology: an overview. In: Denzin NK, 
Lincoln YS, editors. Handbook of qualitative research. Thousand Oaks (CA): Sage; 
1994. p. 273-85. 

Carlbring P, Andersson G, Cuijpers P, Riper H, Hedman-Lagerlof E. Internet-based vs. 
face-to-face cognitive behavior therapy for psychiatric and somatic disorders: an 
updated systematic review and meta-analysis. Cogn Behav Ther. 2018;47(1):1-18. 
Olthuis JV, Watt MC, Bailey K, Hayden JA, Stewart SH. Therapist-supported internet 
cognitive behavioural therapy for anxiety disorders in adults. Cochrane Database Syst 
Rev. 2016(3):CD011565. 


Ontario Health Technology Assessment Series; Vol. 19: No. 6, pp. 1-199, February 2019 194 


References February 2019 


(141) 


(142) 


(145) 


(146) 


(147) 


(148) 


(149) 


(150) 


(151) 


(152) 


(153) 


(154) 


(155) 


(156) 


(157) 


Baumeister H, Reichler L, Munzinger M, Lin J. The impact of guidance on internet-based 
mental health interventions — a systematic review. Internet Interventions. 
2014;1(4):205-15. 

Patten SC. The duration of major depressive episodes in the Canadian general 
population. Chronic Dis Can. 2001;22(1):6-11. 

Patten SB. Recurrence risk in major depression. Depress Anxiety. 2013;30(1):1-4. 
Khan A, Schwartz K. Suicide risk and symptom reduction in patients assigned to placebo 
in duloxetine and escitalopram clinical trials: analysis of the FDA summary basis of 
approval reports. Ann Clin Psychiatry. 2007;19(1):31-6. 

Khan A, Khan S, Kolts R, Brown WA. Suicide rates in clinical trials of SSRIs, other 
antidepressants, and placebo: analysis of FDA reports. Am J Psychiatry. 
2003;160(4):790-2. 

Ferrari AJ, Charlson FJ, Norman RE, Flaxman AD, Patten SB, Vos T, et al. The 
epidemiological modelling of major depressive disorder: application for the Global 
Burden of Disease study 2010. PLoS One. 2013;8(7):e69637. 

Hardeveld F, Spijker J, De Graaf R, Nolen WA, Beekman AT. Recurrence of major 
depressive disorder and its predictors in the general population: results from the 
Netherlands Mental Health Survey and Incidence Study (NEMESIS). Psychol Med. 
2013;43(1):39-48. 

Statistics Canada. Life tables, Canada, provinces and territories (2009 to 2011, no. 5) 
[Internet]. Ottawa (ON): Statistics Canada. 2011. Available from: 


http:/Awww.statcan.gc.ca/access _acces/alternative alternatif.action?loc=http://www.statc 


an.gc.ca/pub/84-537-x/2013005/tbl_ 2009 2011-eng.xlsx 
Mueller Tl, Leon AC, Keller MB, Solomon DA, Endicott J, Coryell W, et al. Recurrence 


after recovery from major depressive disorder during 15 years of observational follow-up. 
Am J Psychiatry. 1999;156(7):1000-6. 

Amick HR, Gartlehner G, Gaynes BN, Forneris C, Asher GN, Morgan LC, et al. 
Comparative benefits and harms of second generation antidepressants and cognitive 
behavioral therapies in initial treatment of major depressive disorder: systematic review 
and meta-analysis. BMJ. 2015;351:h6019. 

Cipriani A, Furukawa TA, Salanti G, Geddes JR, Higgins JP, Churchill R, et al. 
Comparative efficacy and acceptability of 12 new-generation antidepressants: a multiple- 
treatments meta-analysis. Lancet. 2009;373(9665):746-58. 

Williams N, Simpson AN, Simpson K, Nahas Z. Relapse rates with long-term 
antidepressant drug therapy: a meta-analysis. Hum Psychopharmacol. 2009;24(5):401- 
8. 

Mohiuddin S, Payne K. Utility values for adults with unipolar depression: systematic 
review and meta-analysis. Med Decis Making. 2014;34(5):666-85. 

Revicki DA, Wood M. Patient-assigned health state utilities for depression-related 
outcomes: differences by depression severity and antidepressant medications. J Affect 
Disord. 1998;48(1):25-36. 

Mann R, Gilbody S, Richards D. Putting the 'Q' in depression QALYs: a comparison of 
utility measurement using EQ-5D and SF-6D health related quality of life measures. Soc 
Psychiatry Psychiatr Epidemiol. 2009;44(7):569-78. 

Vallejo-Torres L, Castilla |, Gonzalez N, Hunter R, Serrano-Perez P, Perestelo-Perez L. 
Cost-effectiveness of electroconvulsive therapy compared to repetitive transcranial 
magnetic stimulation for treatment-resistant severe depression: a decision model. 
Psychol Med. 2015;45(7):1459-70. 

Kates N, McPherson-Doe C, George L. Integrating mental health services within primary 
care settings: the Hamilton Family Health Team. J Ambul Care Manage. 201 1;34(2):174- 
82. 


Ontario Health Technology Assessment Series; Vol. 19: No. 6, pp. 1-199, February 2019 195 


References February 2019 


(158) 


(159) 


(160) 


(161) 


(162) 


Rosser W, Godwin M, Seguin R. Family medicine research capacity building: five- 
weekend programs in Ontario. Can Fam Physician. 2010;56(3):e94-e100. 

Rosser WW, Colwill JM, Kasperski J, Wilson L. Progress of Ontario's Family Health 
Team model: a patient-centered medical home. Ann Fam Med. 2011;9(2):165-71. 
Association of Family Health Teams of Ontario, Association of Ontario Health Centres, 
Nurse Practitioners' Association of Ontario. Toward a primary care recruitment and 
retention strategy for Ontario: compensation structure for Ontario's interprofessional 
primary care organizations2014 [cited 2018 Apr 04]:[12 p.]. 

Health Quality Ontario. Repetitive transcranial magnetic stimulation for treatment- 
resistant depression: a systematic review and meta-analysis of randomized controlled 
trials. Ont Health Technol Assess Ser. 2016;16(5):1-66. 

Bereza BG, Machado M, Papadimitropoulos M, Sproule B, Ravindran AV, Einarson TR. 
A markov model approach assessing the cost of illness of generalized anxiety disorder 
in Canada. Neurol Ther. 2012;1(1):1. 


Ontario Health Technology Assessment Series; Vol. 19: No. 6, pp. 1-199, February 2019 196 


February 2019 


About Health Quality Ontario 


Health Quality Ontario is the provincial lead on the quality of health care. We help nurses, 
doctors and other health care professionals working hard on the frontlines be more effective in 
what they do — by providing objective advice and data, and by supporting them and government 
in improving health care for the people of Ontario. 


We focus on making health care more effective, efficient and affordable through a legislative 
mandate of: 


e Reporting to the public, organizations, government and health care providers on how the 
health system is performing, 

e Finding the best evidence of what works, and 

e Translating this evidence into clinical standards; recommendations to health care 
professionals and funders; and tools that health care providers can easily put into 
practice to make improvements. 


Health Quality Ontario is governed by a 12-member Board of Directors with a broad range of 
expertise — doctors, nurses, patients and from other segments of health care — and appointed 
by the Minister of Health and Long-Term Care. 


In everything it does, Health Quality Ontario brings together those with first-hand experience to 
hear their experiences and views of how to make them better. We partner with patients, 
residents, families and caregivers to be full participants in designing our programs and services, 
to ensure they are aligned to their needs and priorities. We work collaboratively with 
organizations across the province to encourage the spread of innovative and proven programs 
to support high quality care, while also saving money and eliminating redundancy. And, we work 
with clinicians on the frontlines to use their collective wisdom and experience to bring about 
positive change in areas important to Ontario — such as addressing the challenges of hallway 
health care and mental health. 


For example, 29 Ontario hospitals participated in a pilot program last year that reduced 
infections due to surgery by 18% — which in turn reduces the number of patients 
returning to hospital after surgery and alleviating some of the challenges faced in 
hallway health care. This program enabled surgeons to see their surgical data and how 
they perform in relation to each other and to 700 other hospitals worldwide. We then 
helped them identify and action improvements to care. Forty-six hospitals across Ontario 
are now part of this program, covering 80% of hospital surgeries. 


Health Quality Ontario also develops quality standards for health conditions that 
demonstrate unnecessary gaps and variations in care across the province, such as in 
major depression or schizophrenia. Quality standards are based on the best evidence 
and provide recommendations to government, organizations and clinicians. They also 
include a guide for patients to help them ask informed questions about their care. 


In addition, Health Quality Ontario’s health technology assessments use evidence to 
assess the effectiveness and value for money of new technologies and procedures, and 
incorporate the views and preferences of patients, to make recommendations to 
government on whether they should be funded. 
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Each year, we also help hospitals, long-term care homes, home care and primary care 
organizations across the system create and report on the progress of their annual 
Quality Improvement Plans, which is their public commitment on their priorities to 
improve health care quality. 


Health Quality Ontario is committed to supporting the development of a quality health care 
system based on six fundamental dimensions: efficient, timely, safe, effective, patient-centred 
and equitable. 


Our goal is to challenge the status quo and to focus on long-lasting pragmatic solutions that 
improve the health of Ontarians, enhance their experience of care, reduce health care costs, 
and support the well-being of health care providers. A quality health system results in Ontarians 
leading healthier and more productive lives, and a vibrant society in which everyone benefits. 
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